Former PSA employee AMA on reddit

[Snowman]

Originally Posted by Rhotchkiss

Its noteworthy that the guy states that every card is reviewed by at least two graders (maybe more if "contentious"). So that means that two graders authenticated this 1914 CJ reprint, the T206 EPDG Cy Young Reprint that was discussed here a few months back (dont have pic), and so many altered cards that BODA finds?

Originally Posted by Peter_Spaeth

I don't believe it, myself. The math barely works with one reviewer per card.

Originally Posted by Lorewalker

Maybe on higher grading tiers but my hunch is that even at the $200 level it is one set of eyes per card for a grade to be assigned. There is a verifier but that person simply "looks" at the card once it is in the holder. I suspect 2 secs per card is spent on that part of the grading process.

I agree. I highly doubt that two graders are independently grading every card that comes through their door. Not for the low end stuff. The variance we see in results by submission is too wide for that to be the case. If multiple graders independently graded every card, the variance/inconsistency in grades wouldn't be nearly this wide. And it just doesn't make sense operationally or efficiency wise for the bulk $10/ea ultra-modern type submissions. They're jsut slamming that stuff through. I believe that they think they're grading them twice, but in reality the second "grader" probably just glances at the card and says, "ya, looks good to me".

For the higher end stuff, sure, multiple graders are truly looking at it. I'll buy that. But not the low end cards.

Originally Posted by Johnny630

Isn’t that AI machine grading cards now ? Or is that just high end cards that are being scanned ?

This is definitely not happening. I work in AI/machine learning. I code these algorithms every day for work. I can promise you there's no way in hell they're using "AI" to grade cards. (Nat, if you're reading this and want to know why the hell Genamint isn't working out as well as you had hoped it would, feel free to PM me.)

[Peter_Spaeth]

Originally Posted by Snowman

This is definitely not happening. I work in AI/machine learning. I code these algorithms every day for work. I can promise you there's no way in hell they're using "AI" to grade cards. (Nat, if you're reading this and want to know why the hell Genamint isn't working out as well as you had hoped it would, feel free to PM me.)

Can you shed more light on why AI doesn't work? And clarify whether your opinion applies to modern as well as vintage?

[Rick-Rarecards]

There could be a whole sea of issues. I think of three questions AI/ML could help with
1) Detect if a card is real or fake
2) Classify the card (type, year, etc)
3) Classify the grade

In all of these cases, I can assure you people want to know why the algorithm gave the grade/class/ etc, e.g. explain how the algorithm go to the result. This requires explainable AI, which is beyond what algorithms can do today. Furthermore, all of this requires a large training set (you need a lot of examples) including fake examples! Who has that many training examples sitting around?

[Snowman]

Originally Posted by Peter_Spaeth

Can you shed more light on why AI doesn't work? And clarify whether your opinion applies to modern as well as vintage?

I would be happy to explain, in detail, all of the challenges that something like grading cards faces in the machine learning space. But it'll be a lengthy post and I currently don't hand the bandwidth to write it. Maybe I'll make a separate thread for it when I have time.

[Peter_Spaeth]

Originally Posted by Snowman

I would be happy to explain, in detail, all of the challenges that something like grading cards faces in the machine learning space. But it'll be a lengthy post and I currently don't hand the bandwidth to write it. Maybe I'll make a separate thread for it when I have time.

OK. As I am sure you know a lot of guys have touted it as the solution to all the problems brought out by the scandal, and it seems PSA new ownership has faith in it as well, although perhaps for different purposes. There have been a number of does too does not discussions here but I don't think we've had much contribution from people who really understand the technology. Maybe I missed them.

[Rick-Rarecards]

Originally Posted by Peter_Spaeth

OK. As I am sure you know a lot of guys have touted it as the solution to all the problems brought out by the scandal, and it seems PSA new ownership has faith in it as well, although perhaps for different purposes. There have been a number of does too does not discussions here but I don't think we've had much contribution from people who really understand the technology. Maybe I missed them.

You missed it even in this thread. The short answer is:

There could be a whole sea of issues. I think of three questions AI/ML could help with
1) Detect if a card is real or fake
2) Classify the card (type, year, etc)
3) Classify the grade

In all of these cases, I can assure you people want to know why the algorithm gave the grade/class/ etc, e.g. explain how the algorithm got the result. This requires explainable AI, which is beyond what algorithms can do today. Furthermore, all of this requires a large training set (you need a lot of examples) including fake examples! Who has that many training examples sitting around?

[Peter_Spaeth]

Originally Posted by Rick-Rarecards

You missed it even in this thread. The short answer is:

There could be a whole sea of issues. I think of three questions AI/ML could help with
1) Detect if a card is real or fake
2) Classify the card (type, year, etc)
3) Classify the grade

In all of these cases, I can assure you people want to know why the algorithm gave the grade/class/ etc, e.g. explain how the algorithm got the result. This requires explainable AI, which is beyond what algorithms can do today. Furthermore, all of this requires a large training set (you need a lot of examples) including fake examples! Who has that many training examples sitting around?

I may be misunderstanding you and if so sorry, but is your opinion then that the technology in fact works for 1-3 but the issue is that the result can't be easily explained? Or does that mean as a practical matter it doesn't work?

[Rick-Rarecards]

No problem, it's fun seeing all the application AI/ML can have. Snowman is right its a very long discussion but I will try to give you a 30,000 ft view. You can create 1-3, but they would be very limited. There are technological limitations as well as practical limitations.

The easiest to understand are the practical limitations. So yes, if you can't explain the results the tools are useless. How crazy would the industry be if you received the following letter: "Dear Sir/Madam, our software has determined that your card has a 51% chance of likely being fake. Therefore, we are unable to certify it thank you for using our services."


The reason we can't explain the results are a technical limitation. Current AI/ML is a "blackbox" approach. You have an algorithm and train it on examples. Let's say I was creating an AI/ML tool to do 1) detect if a card is real or not. You basically show the tool a bunch of labeled examples so fake and real cards. It creates its own internal method to determine if a card is fake or real. You then test it on a bunch of cards that it has never seen before and compare its results to graders. If it does a good job you are good to go!

So where do the issues come from? Well if the algorithm has never seen a certain color, or a certain name before, never seen a type of error, there is a weird fleck of dust etc. Characteristics of cards that never existed in the training set (have you seen those cards that had a piece of fabric on them). So, you say well if it encounters something its never seen before it should tell someone to inspect the card! Well, that is an even more complicated problem (anomaly detection). Plus, it can't tell anyone what it didn't understand about the card that broke it (explainable AI). You might even say, well let's jus show it everything that has ever been graded before. This might cause something called overfitting, your algorithm is so fine tuned and specific that it will throw out anything not in its training.

It gets complicated the more you think about it. So this is essentially one of many problems just for the arguably easiest of the 3 problems.





There is no easy checklist to go through for grading a card. Just like with a human grader, you need to have the tool see a bunch of cards So you would say here is an image of a fake card long as you need to explain how you got your results AI/ML won't work.