NonSports Forum

Net54baseball.com
Welcome to Net54baseball.com. These forums are devoted to both Pre- and Post- war baseball cards and vintage memorabilia, as well as other sports. There is a separate section for Buying, Selling and Trading - the B/S/T area!! If you write anything concerning a person or company your full name needs to be in your post or obtainable from it. . Contact the moderator at leon@net54baseball.com should you have any questions or concerns. When you click on links to eBay on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network. Enjoy!
Net54baseball.com
Net54baseball.com
ebay GSB
T206s on eBay
Babe Ruth Cards on eBay
t206 Ty Cobb on eBay
Ty Cobb Cards on eBay
Lou Gehrig Cards on eBay
Baseball T201-T217 on eBay
Baseball E90-E107 on eBay
T205 Cards on eBay
Baseball Postcards on eBay
Goudey Cards on eBay
Baseball Memorabilia on eBay
Baseball Exhibit Cards on eBay
Baseball Strip Cards on eBay
Baseball Baking Cards on eBay
Sporting News Cards on eBay
Play Ball Cards on eBay
Joe DiMaggio Cards on eBay
Mickey Mantle Cards on eBay
Bowman 1951-1955 on eBay
Football Cards on eBay

Go Back   Net54baseball.com Forums > Net54baseball Main Forum - WWII & Older Baseball Cards > Net54baseball Vintage (WWII & Older) Baseball Cards & New Member Introductions

Reply
 
Thread Tools Display Modes
  #1  
Old 08-09-2021, 07:50 AM
Peter_Spaeth's Avatar
Peter_Spaeth Peter_Spaeth is offline
Peter Spaeth
Member
 
Join Date: Apr 2009
Posts: 30,264
Default Grading using artificial intelligence

I think the discussion we were having yesterday is likely to be buried in a thread nominally about something else entirely, so I am starting a new one. Hopefully the posters who weighed in on its deficiencies yesterday will do so here or reproduce their posts, I wasn't comfortable doing that. And hopefully any advocates will weigh in as well. My partially formed opinion based on what I've read here and elsewhere, and heard, is that this technology is a long way from being ready for serious use. And that it isn't likely to help with the problems we all know about, at least any time soon.
__________________
My avatar is a sketch by my son who is an art school graduate. Some of his sketches and paintings are at
https://www.jamesspaethartwork.com/

He is available to do custom drawings in graphite, charcoal and other media. He also sells some of his works as note cards/greeting cards on Etsy under JamesSpaethArt.

Last edited by Peter_Spaeth; 08-09-2021 at 07:51 AM.
Reply With Quote
  #2  
Old 08-09-2021, 08:20 AM
parkplace33 parkplace33 is offline
Drew W@i$e
Member
 
Join Date: Jul 2018
Posts: 1,081
Default

Great topic. Here is my take:

I don’t think we are close nor do I think the grading companies want to seriously use this technology. I liken it to the car tire industry. Car tires typically last 60000 miles. Could a car tire company make a tire that lasts over that amount? Of course they could. But they want you to have to buy new tires often and spend money.

The same applies to grading companies. How many times do we hear “I didn’t get the grade I wanted” so I cracked it out and resubmitted it. If the technology was used properly, cracking and resubmitting cards wouldn’t be possible and thus, grading companies would lose money in the long run.
Reply With Quote
  #3  
Old 08-09-2021, 08:22 AM
Rick-Rarecards Rick-Rarecards is offline
member
 
Join Date: Aug 2021
Posts: 9
Default

Quote:
Originally Posted by Peter_Spaeth View Post
I think the discussion we were having yesterday is likely to be buried in a thread nominally about something else entirely, so I am starting a new one. Hopefully the posters who weighed in on its deficiencies yesterday will do so here or reproduce their posts, I wasn't comfortable doing that. And hopefully any advocates will weigh in as well. My partially formed opinion based on what I've read here and elsewhere, and heard, is that this technology is a long way from being ready for serious use. And that it isn't likely to help with the problems we all know about, at least any time soon.
Pasting the information from yesterday:

I think of three questions AI/ML could help with
1) Detect if a card is real or fake
2) Classify the card (type, year, etc)
3) Classify the grade

In all of these cases, I can assure you people want to know why the algorithm gave the grade/class/ etc, e.g. explain how the algorithm got the result. This requires explainable AI, which is beyond what algorithms can do today. Furthermore, all of this requires a large training set (you need a lot of examples) including fake examples! Who has that many training examples sitting around? Not to mention the level of fidelity needed.

It is a very long discussion but I will try to give you a 30,000 ft view. You can create 1-3, but they would be very limited. There are technological limitations as well as practical limitations.

The easiest to understand are the practical limitations. So yes, if you can't explain the results the tools are useless. How crazy would the industry be if you received the following letter: "Dear Sir/Madam, our software has determined that your card has a 51% chance of likely being fake. Therefore, we are unable to certify it thank you for using our services."


The reason we can't explain the results are a technical limitation. Current AI/ML is a "blackbox" approach. You have an algorithm and train it on examples. Let's say I was creating an AI/ML tool to do 1) detect if a card is real or not. You basically show the tool a bunch of labeled examples so fake and real cards. It creates its own internal method to determine if a card is fake or real. You then test it on a bunch of cards that it has never seen before and compare its results to graders. If it does a good job you are good to go!

So where do the issues come from? Well if the algorithm has never seen a certain color, or a certain name before, never seen a type of error, there is a weird fleck of dust etc. Characteristics of cards that never existed in the training set (have you seen those cards that had a piece of fabric on them). So, you say well if it encounters something its never seen before it should tell someone to inspect the card! Well, that is an even more complicated problem (anomaly detection). Plus, it can't tell anyone what it didn't understand about the card that broke it (explainable AI). You might even say, well let's jus show it everything that has ever been graded before. This might cause something called overfitting, your algorithm is so fine tuned and specific that it will throw out anything not in its training.

It gets complicated the more you think about it. So this is essentially one of many problems just for the arguably easiest of the 3 problems.


So what could todays AI/ML do for detecting a fake card?

I will give you a possible system for 1) detecting a fake card. Assume that the industry agreed on a set of descriptors for how one would fake a card, categories as you will. I'm not familiar with all of the ways to create a fake card so apologies for the limited list: So we could say, 1) Reprint (passing off as original), 2) Washed old card and reprinted, 3) New print, 4) etc.

One could run an algorithm to first tell you if the card is fake or not. Then you could either do the following: have another tool tell you which of the categories is most likely (so pick the single top reason), or it could just give you the likelihood that it thinks the card falls in each of the categories, or you could have individual algorithms for each of the categories and have the system give a probability individually.

Again, these will all be blackbox answers. It won't give you the reason why it picked a certain category over another. It won't tell you which card was washed, AI/ML is not magic! The more detail you want, the more fine tuning hand crafted algorithms you need. Let's not forget, there are always new methods for faking cards so you would have to keep adjusting your algorithms and this means some fakes will always make it through.
Reply With Quote
  #4  
Old 08-09-2021, 08:29 AM
Rick-Rarecards Rick-Rarecards is offline
member
 
Join Date: Aug 2021
Posts: 9
Default

Quote:
Originally Posted by parkplace33 View Post
Great topic. Here is my take:

I don’t think we are close nor do I think the grading companies want to seriously use this technology. I liken it to the car tire industry. Car tires typically last 60000 miles. Could a car tire company make a tire that lasts over that amount? Of course they could. But they want you to have to buy new tires often and spend money.

The same applies to grading companies. How many times do we hear “I didn’t get the grade I wanted” so I cracked it out and resubmitted it. If the technology was used properly, cracking and resubmitting cards wouldn’t be possible and thus, grading companies would lose money in the long run.
Maybe in the future, but the only way to guarantee a card gets the exact same grade would be to store every card and their associated grades in some sort of huge database. The level of detail needed for each card to be able to match up the cracked card with the stored image in the database would be ridiculous. Think of it as using facial recognition software, except you have 50 Mantles who all look alike and the only differences are small changes in some of the "important" characteristics. Systems today are constantly learning, so the score you got last week might change a bit because the system updated its algorithm to account for something new it learned.

Last edited by Rick-Rarecards; 08-09-2021 at 08:31 AM.
Reply With Quote
  #5  
Old 08-09-2021, 08:38 AM
Case12's Avatar
Case12 Case12 is offline
Casey
Member
 
Join Date: Nov 2018
Posts: 681
Default

Some AI SW developers and a HW engineer, if focused on this application, should be able to do this now. It may take them a year or two. Maybe longer to build the database and have experts confirm sampling/interpretation for a while. AI would take over from there (with some bias built in). Probably $2 -3M for pros and outsourcing. Maybe more because AI developers are in very high demand. Probably use Google Tensor Flow and Raspberry PI to start with. Just don't know if the ROI would be worth it.

Just finished consulting with start-up doing image 3D recognition, including movement and sound, then interpretation to animation translation. All wireless between sensors. Stand alone system and/or cloud. That's pretty hard, but they are doing it. I helped them get 10 patents granted on their app so far.

I left them start of pandemic. Yep, it affected me like others. So, I don't know what image probability levels they were going to get. They swore it would be above 95% as AI matured. Their app had to be very close to 100%.

This was a non-profit so ROI was not an issue. Large donors.

Last edited by Case12; 08-09-2021 at 08:47 AM.
Reply With Quote
  #6  
Old 08-09-2021, 08:43 AM
Rick-Rarecards Rick-Rarecards is offline
member
 
Join Date: Aug 2021
Posts: 9
Default

Quote:
Originally Posted by Case12 View Post
Some AI SW developers and a HW engineer, if focused on this application, should be able to do this now. It may take them a year or two. Maybe longer to build the database and have experts confirm sampling/interpretation for a while. AI would take over from there (with some bias built in). Probably $2M for pros and outsourcing. Probably use Google Tensor Flow and Raspberry PI to start with. Just don't know if the ROI would be worth it.

Just finished consulting with start-up doing image 3D recognition, including movement and sound, then interpretation to animation translation. All wireless between sensors. Stand alone system and/or cloud. That's pretty hard, but they are doing it. I helped them get 10 patents granted on their app so far.

I left them start of pandemic. Yep, it affected me like others. So, I don't know what image probability levels they were going to get. They swore it would be above 95% as AI matured. Their app had to be very close to 100%.

This was a non-profit so ROI was not an issue. Large donors.
This in the industry is what we call a magic solution response. It doesn't directly address what the problem is. It speaks about a specific application, uses some indication of high probability, and lets you as the reader generalize this to everything. It doesn't indicate what the app is, what image 3D recognition they are doing. Movement and sound, that wont happen for cards why is it applicable? Google Tensor Flow and Raspberry PI ? So googles general platform and a cheap computer?

Last edited by Rick-Rarecards; 08-09-2021 at 08:46 AM.
Reply With Quote
  #7  
Old 08-09-2021, 08:48 AM
drcy's Avatar
drcy drcy is offline
David Ru.dd Cycl.eback
 
Join Date: Jul 2013
Posts: 3,466
Default

If graders are taking like two seconds to grade most cards (whether or not true, something said in another post), I would think AI would slow that down.
Reply With Quote
  #8  
Old 08-09-2021, 08:57 AM
Case12's Avatar
Case12 Case12 is offline
Casey
Member
 
Join Date: Nov 2018
Posts: 681
Default

Btw, huge database is not what AI is about. It will develop and decide on its own....thus, AI. Large confirmed samples are needed to start. The expert confirmation at beginning is what counts. I don't know what the probabilities are today. 2 years ago they were around 70%. Probably higher now. At minimum AI could be used as a filter to improve efficiency for this app.

Btw, this start-up had the SW and HW up and running. The developers were DOD contractors. They were in the expert confirmation process, which takes time and experts. Also, expert bias is a big concern because AI will grow on its own with that bias. Biggest concern we were concerned with.

All major tech companies have their own AI platform. We were using Google. Raspberry was our starting point, not ending. Just commenting on how to get up and running with new app in short time. Not debating what is best.
.

Last edited by Case12; 08-09-2021 at 09:02 AM.
Reply With Quote
  #9  
Old 08-09-2021, 09:03 AM
Rick-Rarecards Rick-Rarecards is offline
member
 
Join Date: Aug 2021
Posts: 9
Default

Quote:
Originally Posted by Case12 View Post
Btw, huge database is not what AI is about. It will develop and decide on its own....thus, AI. Large confirmed samples are needed to start. The expert confirmation at beginning is what counts. I don't know what the probabilities are today. 2 years ago they were around 70%. Probably higher now. At minimum AI could be used as a filter to improve efficiency for this app.

Btw, this start-up had the SW and HW up and running. The developers were DOD contractors. They were in the expert confirmation process, which takes time and experts. Also, expert bias is a big concern because AI will grow on its own with that bias. Biggest concern we were concerned with.
.
Develop and decide what? What is the task that was being solved? 70% what? Look at GPT-3 how many parameters does it have and how training examples did it need? AI is not magic, you need to be more specific with the claims.

Quote:
All major tech companies have their own AI platform.
Of course and my claim is they overstate their value and abilities.

Last edited by Rick-Rarecards; 08-09-2021 at 09:04 AM.
Reply With Quote
  #10  
Old 08-09-2021, 09:50 AM
Aquarian Sports Cards Aquarian Sports Cards is offline
Scott Russell
Member
 
Join Date: Jun 2016
Location: Pennsylvania
Posts: 6,297
Default

If computers can be good enough for facial recognition in casinos, a living, moving target, I don't understand how they can't be used to, at a bare minimum, recognize a card it has seen before. Also that is what I believe PSA is using Kevin's software for right now. Nat Turner is on record as hating the crack and resubmit game so this makes sense. Of course when things get back to normal it'll be interesting to see if the lost revenue from said game causes a change of heart on the matter.
__________________
Check out https://www.thecollectorconnection.com Always looking for consignments 717.327.8915 We sell your less expensive pre-war cards individually instead of in bulk lots to make YOU the most money possible!

and Facebook: https://www.facebook.com/thecollectorconnectionauctions
Reply With Quote
  #11  
Old 08-09-2021, 10:06 AM
Leon's Avatar
Leon Leon is offline
Leon
peasant/forum owner
 
Join Date: Mar 2009
Location: near Dallas
Posts: 34,196
Default

It is always good to sound like you are really for the hobby when you make so much off of it. I can think of few people that fit that description.

Resubmissions equal millions of dollars (probably). Yeah, as a business owner, lets just get rid of that stream of revenue.

So as AI in general goes, I agree it could take away some revenue. I always heard there have been pills? to make gas out of water but the oil companies always bought out the patent. (probably a tale but sounds reasonable)

Quote:
Originally Posted by Aquarian Sports Cards View Post
If computers can be good enough for facial recognition in casinos, a living, moving target, I don't understand how they can't be used to, at a bare minimum, recognize a card it has seen before. Also that is what I believe PSA is using Kevin's software for right now. Nat Turner is on record as hating the crack and resubmit game so this makes sense. Of course when things get back to normal it'll be interesting to see if the lost revenue from said game causes a change of heart on the matter.
__________________
Leon Luckey

Last edited by Leon; 08-09-2021 at 10:09 AM.
Reply With Quote
  #12  
Old 08-09-2021, 10:08 AM
Rick-Rarecards Rick-Rarecards is offline
member
 
Join Date: Aug 2021
Posts: 9
Default

Quote:
Originally Posted by Aquarian Sports Cards View Post
If computers can be good enough for facial recognition in casinos, a living, moving target, I don't understand how they can't be used to, at a bare minimum, recognize a card it has seen before. Also that is what I believe PSA is using Kevin's software for right now. Nat Turner is on record as hating the crack and resubmit game so this makes sense. Of course when things get back to normal it'll be interesting to see if the lost revenue from said game causes a change of heart on the matter.
I would bet they use gait, speech, height, and other characteristics along with the image.

This is a few years old but its not easy to tell twins apart, https://cacm.acm.org/news/226789-dis...twins/fulltext. You can think of all copies of a card as a twin.
Reply With Quote
  #13  
Old 08-09-2021, 10:11 AM
Peter_Spaeth's Avatar
Peter_Spaeth Peter_Spaeth is offline
Peter Spaeth
Member
 
Join Date: Apr 2009
Posts: 30,264
Default

Quote:
Originally Posted by Rick-Rarecards View Post
I would bet they use gait, speech, height, and other characteristics along with the image.

This is a few years old but its not easy to tell twins apart, https://cacm.acm.org/news/226789-dis...twins/fulltext. You can think of all copies of a card as a twin.
Imagine trying to differentiate among the million or whatever Topps Update Trouts.
__________________
My avatar is a sketch by my son who is an art school graduate. Some of his sketches and paintings are at
https://www.jamesspaethartwork.com/

He is available to do custom drawings in graphite, charcoal and other media. He also sells some of his works as note cards/greeting cards on Etsy under JamesSpaethArt.
Reply With Quote
  #14  
Old 08-09-2021, 10:27 AM
Case12's Avatar
Case12 Case12 is offline
Casey
Member
 
Join Date: Nov 2018
Posts: 681
Default

Quote:
Originally Posted by Rick-Rarecards View Post
Develop and decide what? What is the task that was being solved? 70% what? Look at GPT-3 how many parameters does it have and how training examples did it need? AI is not magic, you need to be more specific with the claims.

Of course and my claim is they overstate their value and abilities.
I agree with everything you say. And AI is not vudu
magic.

In this app we were translating unknown deaf sign language movements to text and voice to animated avatar with text and sound (and visa versa). Some of this is already available without AI. Our goal was the unknown body motions part.

It isn't just talk though They were in the process of developing it. Admitting only hardware prototype (but close enough for design patents and working sound/motion/display hardware) and software prototype (working to the point of needing expert image and motion feedback on known movement). These folk started from scratch and built to that point in two years. I don't know their progress since my leaving in March 2020.

I apologize for overstating. And you are absolutely correct that it depends on the app. The one they were doing was pretty tough though. (Unknown motion, image, sound, text, context).
Reply With Quote
  #15  
Old 08-09-2021, 11:32 AM
mrreality68's Avatar
mrreality68 mrreality68 is offline
Jeffrey Kuhr
Member
 
Join Date: Jul 2017
Location: Long Island, NY
Posts: 5,607
Default

Hi

I do not know enough about AI to know where the technology is at or not at. But I cannot believe we already do not have the technology with all the advances that are mentioned and the things AI already does things like

From Forbes Magazine

It can produce breathtaking original content: poetry, prose, images, music, human faces. It can diagnose some medical conditions more accurately than a human physician. Last year it produced a solution to the “protein folding problem,” a grand challenge in biology that has stumped researchers for half a century.

Not to mention that they already automating business processes, gaining insight through data analysis, and engaging with customers and employees.

So why would it not be able to do cards unless it is as someone mentioned all about the income stream of Cracking the cases to regrade?
__________________
Thanks all

Jeff Kuhr

https://www.flickr.com/photos/144250058@N05/

Looking for
1920 Heading Home Ruth Cards
1933 Uncle Jacks Candy Babe Ruth Card
1921 Frederick Foto Ruth
Joe Jackson Cards 1916 Advertising Backs
1910 Old Mills Joe Jackson
1914 Boston Garter Joe Jackson
1915 Cracker Jack Joe Jackson
1911 Pinkerton Joe Jackson
Shoeless Joe Jackson Autograph
Reply With Quote
  #16  
Old 08-09-2021, 03:30 PM
Peter_Spaeth's Avatar
Peter_Spaeth Peter_Spaeth is offline
Peter Spaeth
Member
 
Join Date: Apr 2009
Posts: 30,264
Default

Quote:
Originally Posted by mrreality68 View Post
Hi

I do not know enough about AI to know where the technology is at or not at. But I cannot believe we already do not have the technology with all the advances that are mentioned and the things AI already does things like

From Forbes Magazine

It can produce breathtaking original content: poetry, prose, images, music, human faces. It can diagnose some medical conditions more accurately than a human physician. Last year it produced a solution to the “protein folding problem,” a grand challenge in biology that has stumped researchers for half a century.

Not to mention that they already automating business processes, gaining insight through data analysis, and engaging with customers and employees.

So why would it not be able to do cards unless it is as someone mentioned all about the income stream of Cracking the cases to regrade?
I think it's perhaps a false equivalency, sort of like asking if we can put a spaceship on Mars why can't we cure the common cold? It may be very well suited for the tasks you mention but that doesn't necessarily translate. PS no insult to anyone here but that it can diagnose better than a human doctor, based on my recent experiences, doesn't impress me.
__________________
My avatar is a sketch by my son who is an art school graduate. Some of his sketches and paintings are at
https://www.jamesspaethartwork.com/

He is available to do custom drawings in graphite, charcoal and other media. He also sells some of his works as note cards/greeting cards on Etsy under JamesSpaethArt.

Last edited by Peter_Spaeth; 08-09-2021 at 03:32 PM.
Reply With Quote
  #17  
Old 08-11-2021, 01:46 AM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

Part 1 of 3...

I apologize in advance for the ridiculously long string of posts that this is going to be, but I did warn you that this is a lengthy discussion, so I guess I'm at least partially delivering on that promise lol.

There is a lot of confusion here, and across the hobby, about what the term "AI" means and how it works. I'll try to clear up some of this confusion as this is my field of expertise. I write "AI" or "ML" code for work every day and have a strong understanding of how these algorithms work, why they work, where they are likely to go wrong and why.

First, I have to point out that while 'Artificial Intelligence' and 'Machine Learning' are perhaps kissing cousins, they're not quite the same thing. The problem of grading cards with computational algorithms is a 'Machine Learning' problem, not an 'Artificial Intelligence' problem. Artificial Intelligence is when you have recursive algorithms that make use of something like deep learning or ensemble models with deep neural networks where the algorithms can often learn from themselves. This would be something like Alpha Go or Alpha Zero (the best chess "player" in the world) or Deep Mind, or Tesla's self-driving cars that learn how to drive better through simulations of millions of miles. The people coding these algorithms set up the framework and outline the "rules" for it to be able to learn on its own and then sorta turns it loose on the problem. This is not applicable to the challenge of grading cards.

Grading cards is a 'Machine Learning' problem. Specifically, computer vision and classification. In order to understand why grading cards is not a problem well suited for machine learning, you have to first understand how these algorithms work and what their limitations are because there are some important limitations that I would argue render this technology borderline useless for the specific application of grading cards. I will go into this in more detail below, but for now, I'm just pointing out that I take issue with using the term 'AI' for grading. This is NOT an AI problem. It is a machine learning problem. However, "AI" sounds cooler, so "AI grading" it is, right? This is a marketing ploy. OK, I digress.


How computer vision works:

Imagine you have a photo of you and your family sitting down in the park having a picnic, surrounded by fields of green grass. Try to visualize the photo. You know how that image looks to you, but what does it "look like" to a computer? Everyone is familiar with the binary 1s and 0s at the operating system level of a computer, but let's pull back from that and try to interpret how a computer might see color and objects in a photo. Most of you are probably familiar with RGB colors, but if not, it's helpful to know that colors on your computer screen can be rendered using RGB color values, which range from 0 to 255 for Red, Green, and Blue. So every tiny little pixel in your family photo can actually be defined by it's RGB color values. Those pixels in your photo that are part of the green grass surrounding you all look something like [0, 255, 0] meaning 0 parts red, 255 parts green, and 0 parts blue. Now imagine how that entire photo could be represented in a giant map mathematically. Break it down into 3 different matrices or grids: one for red values, one for green, and one for blue. The matrix for red would have a ton of ~0s in it (pretty much everywhere that the green grass is located) but would have higher values in the center of the matrix which correspond to where the people are sitting since people have red color tones. The green matrix would have a ton of ~255 values in it since there is grass all around you, but it would have lower values in the center where the people are since people aren't green. Make sense?

OK, so now we have 3 different matrices, or think of them as maps if that helps, where each pixel from the photo has a corresponding color value. These matrices full of numeric values are what enable computers to "learn" from photos. What sorts of things can a computer learn from an image? Quite a lot actually. One of the primary ways a computer can tell that something is different about a particular section of a photograph is through something known as "edge detection". Edge detection makes use of some fancy math to identify where the edges of an object are located in the photo. So in our example here, one of the "edges" would be where the green grass meets up against the people in the center of the photo. The mathematical values are different on each side of this "edge", which helps the computer to detect that this is an important location in the photo, and it learns to pay attention to it. Make sense? Great. If not, well, I apologize for being a crappy teacher. But this is the gist of how computers see an image and how they use mathematics to identify key aspects of a photo (or a scan in the case of grading cards). If you're perceptive and you can visualize how these matrices of numbers might look to the ML algorithms, you can probably already see how this could be problematic for grading cards. I'll get into that below. It's a pretty lengthy discussion though.


How machine learning classification models work:

One of the most common machine learning problems that data scientists work on is building classification models which aim to classify (or "categorize", or "label") something as belonging to a particular class. A simple example of this might be to build a model to predict whether or not someone is Male or Female based on a set of attributes that the computer learns from. So we might train that algorithm by feeding it the height, weight, hair length, ring finger to middle finger length ratios, hip measurements, shoe size, eyelash length, how fast they can run, and how much time they spend each month shopping or talking on the phone (stereotypes be damned). Then we feed all of that data to the algorithm for each person and tell the computer whether that person is a male or a female. The computer would then learn what each profile looks like and would be able to provide probabilistic estimates of someone being a male or a female for any new data you threw at it. So a person who is 6'2", 195 lbs with a size 12 shoe that runs a 5.1-second 40-yard dash with medium length hair might get classified as having something like an 81% probability of being a male and a 19% probability of being a female according to the algorithm (note that these algorithms are almost never quite as confident as you might want them to be). In addition to binary classes like 'Male' and 'Female', or 'yes' and 'no', or 'true' and 'false' type problems, there are also what are known as multi-class classification problems. So this might be something like classifying whether an animal is a fish, bird, mammal, reptile, or amphibian. The output for a model like this might be something like 3% probability of being a fish, 12% bird, 7% mammal, 46% reptile, and 32% amphibian if you were feeding it with the data of a monitor lizard. Multi-class classification problems are much less performant than binary classification problems for obvious reasons. More options lead to more variance leads to more uncertainty, which equals more errors made by the computer when classifying.
Reply With Quote
  #18  
Old 08-11-2021, 02:26 AM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

Part 2 of 3...


OK, great. Got it, how does that apply to grading cards?

The problem of grading cards can be broken down into several distinct classification problems. This could be classifying a card as 'authentic' vs 'counterfeit', or 'trimmed' vs 'not trimmed', or 'recolored' vs 'not recolored', or 'creased' vs 'not creased', etc. However, the problem of assigning a paricular grade is a multi-class classification problem. Theoretically, the computer would "learn" how to recognize what a 10 looks like, what a 9 looks like, an 8, a 7, ..., so on and so forth. Then it would assign probability estimates for each grade. In practice, this would probably look something like 1% 1, 1% 2, 2% 3, 5% 4, 7% 5, 13% 6, 18% 7, 30% 8, 16% 9, 7% 10, after which the card would be classified as an 8 since the highest likelihood associated with it was an 8 at 30%. However, you could think of it as only being 30% confident that it's an 8. That's not super helpful in practice, and it's precisely the type of output we would expect from a mutli-class classifier like this. And this is IF it's working well despite all the other numerous challenges which I'll outline below. As Rick-Rarecards pointed out, there are both technical challenges and practical challenges that prevent this from being a useful application for "AI" or machine learning. I believe these challenges render this problem borderline futile and at minimum a considerable waste of resources. I will outline these challenges as I see them below.

Technical challenges

Identifying a trimmed card:

One of the most important challenges to understand is what's required in order to build what we call a "training data set" that the computer can learn from. Let's take the problem of detecting a trimmed card as an example. In order for a computer to be able to classify a card as either 'trimmed' or 'not trimmed', it first needs to learn what trimmed cards look like as well as what non-trimmed cards look like. How does it accomplish this? It uses computer vision as I talked about above, leveraging convolutional neural networks and "edge detection" ("edges" in an image like where the grass meets the player in a photo as I described above, not the physical edges of a card) to detect anomolies in an image or scan. So you create a large database of images (thousands and thousands of images at minimum) where each image is labeled as being either trimmed or not trimmed. Where do these labels come from? Humans, of course (and likely the graders specifically). So they have to sit down and document millions of cards, one by one, marking it as trimmed or not trimmed, recolored or not recolored, scratched or not scratched, creased or not creased, etc. Pretty much anything you want to be able to detect about a card, they have to have a massive database to learn from and humans have to physically examine those cards and label the training datasets. Then you feed that dataset to the machine learning algorithms (or rather a highly skilled, and extremely expensive, team of data scientists does this) and after a bit of black magic, you then have a computer that is "capable" of identifying trimmed cards. I put the word 'capable' in quotes here because there's a major caveat that needs to be understood here, and this is perhaps the biggest issue of them all. The machine learning algorithms, without question, will not be as good at humans at detecting trimmed cards. In fact, they won't even be remotely close to as good as humans are. Here's the problem. First, remember that training data set that we created for the ML algorithms to learn from? Well, humans labeled it! So it's not going to learn what a trimmed card looks like, it's going to learn what a trimmed card THAT HUMANS ARE CAPABLE OF DETECTING IN THE FIRST PLACE looks like. Remember, there are countless trimmed cards that humans can't detect. If it passes through a human undetected and gets fed to the algorithm, the algorithm is told that this is what a non-trimmed card looks like, despite the fact that it was actually trimmed. But even worse than that, the algorithm is working off of a database of scanned images. Even if the scan is extremely high definition, it's still just an image from one single angle of the card, looking straight at it. Imagine if it were your job to detect trimmed cards, but you weren't allowed to hold them and feel the edges with your fingers, or even hold them and just rotate the card in hand at different angles, catching how the light bounces off the card with every rotation. All you had to work with was a scanned image of the card on your computer screen. If you think you'd be good at detecting trimmed cards just by looking at a scanned image of it, I promise you you're wrong. You might be great at detecting a botched trim job, but nobody can detect a good one. And if you think measuring a card always (or even often) tells you whether or not a card has been trimmed, again, you're wrong. It's plausible that an ML algorithm could be trained to detect a trimmed edge on a vintage card that has 3 frayed edges and one super straight, clean, smooth edge. But those aren't exactly difficult to detect to begin with, so this isn't much of a win.

Also, you can't just create one large training dataset of all cards. You have to have separate training data and separate ML models for all different types of cards, each training dataset requiring many thousands of cards at minimum, and likely hundreds of thousands of cards to be even remotely performant (good luck with that). All of these are labeled individually by hand. You can't train a model on 1950s Topps cards and then scan a 2018 Topps Chrome Shohei Ohtani and expect it to know if the Ohtani has been trimmed. It will say it's trimmed every single time because the Ohtani has sharp edges and the 1950s Topps don't. A grader knows to differentiate this, a computer doesn't. You would need to have separate datasets for each card type. And this is just the tip of the iceberg. Trying to teach an algorithm to detect trimmed cards is a fool's errand. And if you think all of these hurdles can be overcome simply by scanning every card with some sort of 360 degree spherical scans, LOL. Ya, good luck with that too. You'd then need a separate ML model for every single angle, and now your problem just exploded 1,000 fold.

Detecting a recolored card:

This one is tricky, although it is perhaps the most interesting project to work on of all the possible ML applications for detecting altered cards. But it's an insanely large problem to solve. Here, you'd likely need far more training data sets than you would even in the trimmed cards problem. There's more variation in card printing techniques, inks, surfaces, and especially images within the actual cards themselves than there is variation in card stocks or edges like in the trimmed cards example. Here, you'd almost have to have a separate training set at least for every single issue (so separate training data for 1952 Topps and another for 1933 Goudey, one for T206, etc.), and depending on performance, you may even need one for each individual card! And remember, every training set requires, at minimum, many thousands of cards scanned, but likely hundreds of thousands to be even remotely performant. So the more granular your requirements become, the less plausible this problem is to solve. But let's pretend for a moment that we could at least group together certain cards. Perhaps all 1952 through 1956 Topps cards could be used in one training set. Every time the ML algorithm sees a print defect, it's going to think that is suspicious and will flag that card as having a high likelihood of being recolored. And again, this suffers from the same problem as the trimmed training data in that it can only learn to detect cards that humans can already detect as being recolored to begin with. I suspect it would perform quite well at picking up the obvious recoloring jobs, but then again, do we really need help with those? Not likely. The hope would be that it would be able to identify cards that were recolored very subtly, ones that might slip through human grading, but if humans can't flag those to begin with, it won't be able to learn what they look like because the training data doesn't flag them as recolored. It says they're not recolored. But even if all the cards were correctly labeled in the training data, it still would have a very difficult time distinguishing between print defects, a piece of lint on the scanner bed, a damaged card, one that was in fact recolored, and even cards which just have abnormalities in the image itself. Especially with modern cards. This could be very problematic. Basically, any abnormality in the image could result in that card being flagged as having a high probability of being recolored. And if you tuned the algorithms to not be as sensitive to these abnormalities, then there would be a tradeoff that would result in more recolored cards not getting flagged. There are always tradeoffs in machine learning.

As a reference for how these algorithms work, and what level of performance one might hope for, there was a somewhat infamous competition on a popular data science website several years back that had machine learning experts all over the world competing to come up with the best algorithm to be able to detect whether a picture was of a cat or a dog. The winner was able to code an algorithm that was 97% accurate. On one hand, 97% sounds pretty impressive, but when you weigh its performance against a human, who would get it right ~100% of the time, then it's no longer all that impressive. These algorithms are great for automating away large problems where we just don't have the manpower to be able to go through every photo manually, one at a time. So if we had millions of photos to classify as cat or dog, and we didn't have the time or manpower needed to do it manually, and if a 3% error rate was acceptable, then it would be a huge win, potentially saving some company millions of dollars in costs. But for the problem of grading cards, you can't accept a 3% error rate on a problem as simple as this. And that's just for detecting if an image is a cat or a dog. If you're trying to detect something as challenging as trimming or recoloring, the error rates would be much, much higher. When it comes to grading cards, the need for accuracy far outweighs the need/benefits of automation.

Detecting edge and corner wear:

For a set like 1986 Fleer basketball that has deep red borders and white paper stock, an ML algorithm could probably learn how to identify what good corners look like and what soft or bad corners look like because the matrices would show clear edge detection differences where the red borders and soft white corners meet in the image. For this problem, edge detection "works". Same with 1971 Topps baseball and the black borders. It could easily detect white chipping along the edges of those cards as it would show up in the data of the matrix. However, take a card with white borders and white paper stock and scan the image and you can quickly see how the algorithms would fail to identify the chipping or bad corners because the scanned image does not have an "edge" to detect (white on white doesn't create an "edge" in an image that can be represented mathematically). For this reason, "AI grading" would certainly underperform the expectations/needs of any TPG.

Detecting surface issues:

If you've made it this far into my post, then I'm guessing you're already able to anticipate what the issues might be for this problem. First of all, many surface issues wouldn't even show up in a scan because a scan is only taken at one angle, and you often have to rotate a card at just the right angle to be able to see surface flaws. How many times have you bought a raw card on eBay that looked perfect even in a zoomed-in scan, only to have it show up with surface flaws or even wrinkles? It happens all the time because scans often don't pick up on these flaws, especially with modern chrome cards. Feed that image to an ML algorithm and it won't be able to see it either. But even if it could, it still would need to be able to differentiate between a surface flaw and just some random abnormality in the image itself. It would also need to be able to differentiate between a surface issue and the natural variation in paper stock for vintage cards. 1948 Leaf cards come to mind as those often have dark paper fibers that are visible even through the print. Also, think of the image in the card itself. Is that little speck a raindrop in the photo or a flaw? Is it from dirt on the photographer's lens? Is it a scratch in the surface? Is it lint on the scanner bed? A scratch on the scanner bed? You get the point. All of these things cause an increase in the error rates that an ML algorithm would produce. And again, it would underperform any human that's even remotely competent.

I should also point out that each of the use cases above are the ones that the TPGs are MOST hopeful about lol. In a recent interview, Nat Turner explained that they are not currently using Genamint technology specifically for grading cards, and that realistically, they probably only hope to be able to use it to identify altered cards or cards that have already been submitted for grading before. And even this lofty goal they aren't planning to achieve until the end of the year. "AI grading" isn't coming to PSA anytime soon, and I'll go ahead and go out on a limb here and say it likely never will. The problem of actually assigning a numerical grade to a card is considerably more challenging than any of these binary classification models above, and would produce considerably higher error rates than any of the issues above. It's just not an ideal application for machine learning. For some tasks, computers can be taught remarkably well how to do something. But it's not a magic solution for everything, and you really need to understand the problem you're trying to solve deeply AND have a deep knowledge of how these algorithms work to begin with in order to know if you have a problem that is well suited for machine learning.

This is what happens when executives get excited about technology that they don't understand and buzzwords like "AI" simply because everyone else is doing it, so why shouldn't they? It seems like everyone and their brother in the corporate world today thinks "AI" is coming to revolutionize their industry and that they just have to win the race and get there before their competitors do. But in reality, many of the problems they're trying to solve just aren't well-suited applications for machine learning. Hell, even Uber and Lyft both gave up on their automated driving projects, and that's a problem that is extremely well-suited for AI. There are no shortage of problems that AI and ML are going to solve in the near future, or industries that will be disrupted by these technologies. Grading cards just isn't one of them.

Last edited by Snowman; 08-11-2021 at 02:54 AM.
Reply With Quote
  #19  
Old 08-11-2021, 02:46 AM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

Part 3 of 3...


Practical limitations:

Classification models such as these would come with extremely high error rates, and these types of ML algorithms are simply going to get it wrong far too often because of the natural variation that exists across different card types and card images and the limitations on the input data (high def scans only show so much). The confidence intervals that would accompany an ML model's predictions would be so wide that they would be borderline useless in practice. As Rick-Rarecards pointed out earlier, a model that outputs a 51% chance of your card being authentic just isn't very useful, and it's precisely the type of output that these types of models produce.

The data warehouse required just to support these types of projects is extremely expensive to build and even more expensive to maintain. Data scientists (or "AI/ML engineers", there are many names for these types of jobs) are also not cheap. Neither are the systems architects and database admins necessary for supporting them. Most skilled data scientists' salaries could probably cover 3 or 4 of the most experienced graders on a TPGs payroll, and they'll need an entire team of them to work on these problems. Also, the problems themselves just aren't all that interesting to work on from a problem-solving perspective, and good data scientists are often more motivated by being able to work on something novel and exciting than they are by just salary alone since any good data science job pays well and there's no shortage of interesting companies working on fun and interesting problems that are all competing for their talents. Keeping the good ones around won't be easy. Especially in California. Overall, the scale of the investment in such a project is difficult to exaggerate. The money PSA spent to acquire Genamint is just the tip of the iceberg. The juice will not be worth the squeeze. I'd wager everything I own on that.

TPGs already have enough boxes to check in their pipeline before a card gets returned to the customers. Receiving, research, grading, QC, and shipping. If they were to add taking high-definition scans and running numerous ML algorithms for each card to that pipeline, it could easily triple the time spend on every card. And for what benefit? Worse predictions than humans can already do? Maybe they hope to flag cards to potentially examine more closely? And what percentage of those would be false flags? A lot, that's for sure. Also, I guarantee it would just turn into a running joke with the graders. They'll just roll their eyes every time a math nerd comes up to them with a "questionable card" report. They'll quickly realize on day one that they are better at detecting this stuff than the algorithms are. It's just not practical. It's not the solution they were hoping for.

Also, who would be responsible for interpreting the models' outputs? Normally, this would be a data scientist who interprets model results for executives at other companies. Their expertise is needed to explain some of the anomalies, which there will be no shortage of. But to pay someone with that skill set just to interpret model results on every single card that comes through a TPG like PSA? Yikes. That's a pretty big ask. And if you had a non-skilled worker doing it, then you might as well just scrap the entire project.


What it could do well:

ML could be used to grade centering on all cards with a clearly defined border. It would be a fairly straightforward model to build and one that would be expected to perform well. Yay, I guess? How big of a win is this really though? Do you really need a machine learning model to tell you that a bordered card is off-centered?

However, non-bordered cards pose a much more challenging problem. You could train a model to learn centering by having it pay attention to how far from the edges the Topps logo is for a particular set, but then you're running into the problem again of having to build an entire dataset with tens of thousands of cards from just one particular set (something they rarely have) in order to create the training data it needs to learn how to identify what a well-centered card looks like from say 2022 Topps Chrome (or any other new set that it hasn't seen yet). This is just not practical. And you can't combine different sets with different logo locations into the same training data, because one logo might be well centered 3/16" from the top and left edges whereas a different logo for a different set might be well centered 1/2" from the top and left edges. Having non-uniform distances both being "well-centered" would confuse the algorithm.

Someone mentioned that ML could be used to identify which set a card was from, perhaps to help in the research stage of a TPGs pipeline. In theory, this is possible, but I'm not so sure this is an "ML" problem per se. You certainly wouldn't build a multiclass classification model with tens of thousands of different classes (here a 'class' would be a set of cards like 1987 Topps or 2019 Topps Chrome Sapphire Edition, etc.) because that's just way too many classes for a problem like this. I suppose you could try a different approach, but it seems like more of a matching algorithm type problem, not really a machine learning one that would require a training set of data to learn from.

Fingerprinting cards - Again, this isn't really "AI" or "ML". This is just a matching algorithm. Just like when the FBI "runs someone's prints" for a fingerprint match or dental records. You're basically comparing the numeric values in the RGB (or similar) matrix I mentioned earlier against other image files in a database and calculating something like the Euclidean distance between all vectors in the matrix to come up with a similarity score. When two images have extremely low, or near zero distance, they're probably a match. But again, this is just math and some basic coding skills, this is not machine learning and certainly not "AI" (although I suppose they are somewhat related fields).


Additional challenges that I didn't address:
- Autographs on cards
- Memorabilia cards
- Short printed cards that don't have enough copies to be able to create training datasets from
- Some cards are bowed, others are flat, this could distort the "edge detection" locations in scanned images
- Crossover submissions with cards currently in other slabs
- And a whole lot more...
Reply With Quote
  #20  
Old 08-11-2021, 06:29 AM
vintagetoppsguy vintagetoppsguy is offline
D@v!d J@m3s
Banned
 
Join Date: Sep 2009
Location: Houston, TX
Posts: 5,981
Default

Quote:
Originally Posted by Leon View Post
I always heard there have been pills? to make gas out of water but the oil companies always bought out the patent. (probably a tale but sounds reasonable)
You're thinking of an episode of the Munsters...

https://www.youtube.com/watch?v=Pnsw9Q2RV-E

Reply With Quote
  #21  
Old 08-11-2021, 06:55 AM
mrreality68's Avatar
mrreality68 mrreality68 is offline
Jeffrey Kuhr
Member
 
Join Date: Jul 2017
Location: Long Island, NY
Posts: 5,607
Default

Quote:
Originally Posted by vintagetoppsguy View Post
You're thinking of an episode of the Munsters...

https://www.youtube.com/watch?v=Pnsw9Q2RV-E

Drop The Mike.

Point Made
__________________
Thanks all

Jeff Kuhr

https://www.flickr.com/photos/144250058@N05/

Looking for
1920 Heading Home Ruth Cards
1933 Uncle Jacks Candy Babe Ruth Card
1921 Frederick Foto Ruth
Joe Jackson Cards 1916 Advertising Backs
1910 Old Mills Joe Jackson
1914 Boston Garter Joe Jackson
1915 Cracker Jack Joe Jackson
1911 Pinkerton Joe Jackson
Shoeless Joe Jackson Autograph
Reply With Quote
  #22  
Old 08-11-2021, 07:37 AM
maniac_73's Avatar
maniac_73 maniac_73 is offline
CostA Kl@d1@n0s
Member
 
Join Date: May 2012
Location: Santa Clara, Ca
Posts: 625
Default

Snowman, I've worked in Sports Technology for 20 years and that was the best breakdown of AI/ML that I have ever read. Thank you for that!
Reply With Quote
  #23  
Old 08-11-2021, 07:45 AM
abothebear abothebear is offline
George E.
Member
 
Join Date: Feb 2011
Location: Birmingham, Alabama
Posts: 644
Default

Here is what I would want out of a third-party grade. I realize that the populace does not want this, but it makes the most sense to me as one who wants to buy and sell online (where the seller cannot have them in hand to check for themselves).

I'd like a verdict on five or six categories (like what Becket does), but with no composite "score." I realize the composite score is the main driver for the market, people want their card ranked, and they want the math done for them so they can price point it. But I just need a third party to tell me what I can't see for myself (centering is not necessary but could easily be measured and a ratio included, so why not add that). But clarity and color I can see from the scans, I don't need a TPG to add that into their composite score. For "Altered" there should be two only two possible answers - Signs of Altering detected or No signs of Altering detected.

And I'd like the grading levels were broader (not a 10 scale, maybe take one or two levels out of the poor to mint scale).

Real: Y
Trimmed: No
Altered: No signs found
Centering Front: Vert60/40, Hor58/42
Centering Back: Vert67/33, Hor28/72
Corners: Very Good
Edges: Near Mint
Surface texture front: Fair
Surface texture back: Good

Again, I realize that in the market the score is everything, and this is a pipe dream, but as a buyer, this is what I want to see, and all I want to see. I want to look at the scans, verify what I see or don't see against the TPG readings, and go about my business.
Reply With Quote
  #24  
Old 08-11-2021, 11:15 AM
AndrewJerome's Avatar
AndrewJerome AndrewJerome is offline
Member
 
Join Date: Apr 2009
Posts: 295
Default

Snowman,

Just wanted to thank you for the incredibly detailed breakdown. The information you provided is a fascinating read for us "card dorks."

Andrew
__________________
callmefugazi@yahoo.com
www.slackjobcards.com
Reply With Quote
  #25  
Old 08-11-2021, 11:22 AM
Johnny630 Johnny630 is offline
Johnny MaZilli
Member
 
Join Date: Nov 2015
Posts: 4,145
Default

Big Thank You To Snowman !!! Really informative
Reply With Quote
  #26  
Old 08-11-2021, 02:25 PM
Peter_Spaeth's Avatar
Peter_Spaeth Peter_Spaeth is offline
Peter Spaeth
Member
 
Join Date: Apr 2009
Posts: 30,264
Default

Snow, very much appreciate the time you took. Other than that Mrs. Lincoln how was the play.

This should put a stop to all the lofty talk we've heard from people about how AI is going to be a game changer.
__________________
My avatar is a sketch by my son who is an art school graduate. Some of his sketches and paintings are at
https://www.jamesspaethartwork.com/

He is available to do custom drawings in graphite, charcoal and other media. He also sells some of his works as note cards/greeting cards on Etsy under JamesSpaethArt.

Last edited by Peter_Spaeth; 08-11-2021 at 02:28 PM.
Reply With Quote
  #27  
Old 08-11-2021, 02:27 PM
mrreality68's Avatar
mrreality68 mrreality68 is offline
Jeffrey Kuhr
Member
 
Join Date: Jul 2017
Location: Long Island, NY
Posts: 5,607
Default

Quote:
Originally Posted by Peter_Spaeth View Post
Snow, very much appreciate the time you took.
+1 Great job and great detail. Better understanding now. Long way to go
__________________
Thanks all

Jeff Kuhr

https://www.flickr.com/photos/144250058@N05/

Looking for
1920 Heading Home Ruth Cards
1933 Uncle Jacks Candy Babe Ruth Card
1921 Frederick Foto Ruth
Joe Jackson Cards 1916 Advertising Backs
1910 Old Mills Joe Jackson
1914 Boston Garter Joe Jackson
1915 Cracker Jack Joe Jackson
1911 Pinkerton Joe Jackson
Shoeless Joe Jackson Autograph
Reply With Quote
  #28  
Old 08-11-2021, 03:04 PM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

Quote:
Originally Posted by Peter_Spaeth View Post
Snow, very much appreciate the time you took. Other than that Mrs. Lincoln how was the play.

This should put a stop to all the lofty talk we've heard from people about how AI is going to be a game changer.
I'd be a poor data scientist if I didn't also point out this this is just my opinion. I could be wrong, and if so, it wouldn't be the first time I was proven wrong by someone more creative and intelligent than myself who found a new method of solving a difficult problem that changes the game for everyone. That said, I'd gladly wager a lot of money against it.
Reply With Quote
  #29  
Old 08-11-2021, 03:06 PM
Peter_Spaeth's Avatar
Peter_Spaeth Peter_Spaeth is offline
Peter Spaeth
Member
 
Join Date: Apr 2009
Posts: 30,264
Default

Quote:
Originally Posted by Snowman View Post
I'd be a poor data scientist if I didn't also point out this this is just my opinion. I could be wrong, and if so, it wouldn't be the first time I was proven wrong by someone more creative and intelligent than myself who found a new method of solving a difficult problem that changes the game for everyone. That said, I'd gladly wager a lot of money against it.
Well, the GenaMint guy has posted here, let's see if he has a rebuttal. I'm very impressed though with the force of your arguments and the clarity with which you made them.
__________________
My avatar is a sketch by my son who is an art school graduate. Some of his sketches and paintings are at
https://www.jamesspaethartwork.com/

He is available to do custom drawings in graphite, charcoal and other media. He also sells some of his works as note cards/greeting cards on Etsy under JamesSpaethArt.

Last edited by Peter_Spaeth; 08-11-2021 at 03:09 PM.
Reply With Quote
  #30  
Old 08-11-2021, 04:05 PM
maniac_73's Avatar
maniac_73 maniac_73 is offline
CostA Kl@d1@n0s
Member
 
Join Date: May 2012
Location: Santa Clara, Ca
Posts: 625
Default

Quote:
Originally Posted by Snowman View Post
I'd be a poor data scientist if I didn't also point out this this is just my opinion. I could be wrong, and if so, it wouldn't be the first time I was proven wrong by someone more creative and intelligent than myself who found a new method of solving a difficult problem that changes the game for everyone. That said, I'd gladly wager a lot of money against it.
I think you are right. It technically can be done but the insane cost and resources to develop are not worth the ROI like Self Driving cars.
Reply With Quote
  #31  
Old 08-11-2021, 04:32 PM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

Quote:
Originally Posted by maniac_73 View Post
I think you are right. It technically can be done but the insane cost and resources to develop are not worth the ROI like Self Driving cars.
Yes, you could definitely write algorithms that could assign grades to cards and detect alterations. I just don't think the output of such models would be even remotely accurate enough to be all that helpful or useful in the real world. Would be happy to engage in a discussion with the genamint guys if they wanted to offer a rebuttal though. Also worth noting is that there are other ML approaches that could be used, but they'd all be subject to the same challenges.

There's a reason why PSA isn't currently using this technology to grade cards though despite having made a significant investment in it several months ago.

My suspicion is that they're simply taking scans in high def and with other light spectrums like UV and writing software to detect card stock that glows in blacklight, and that fingerprints cards and runs them against their database of existing cards to see if they have a match. Then those cards can be flagged for further investigation by the grading team. But this isn't machine learning, and it's certainly not "AI". I suspect they're just being rather liberal with the term "AI grading" because it's a big buzzword these days and it sounds good in a marketing campaign.
Reply With Quote
  #32  
Old 08-11-2021, 04:51 PM
BobC BobC is offline
Bob C.
 
Join Date: Apr 2009
Location: Ohio
Posts: 3,275
Default

Quote:
Originally Posted by Snowman View Post
I'd be a poor data scientist if I didn't also point out this this is just my opinion. I could be wrong, and if so, it wouldn't be the first time I was proven wrong by someone more creative and intelligent than myself who found a new method of solving a difficult problem that changes the game for everyone. That said, I'd gladly wager a lot of money against it.
Agree with you. As you pointed out in your detailed narrative, there will always be a need for human involvement and input in regards to AI and ML, and that is the biggest problem with just turning everything over to machines and computers. The next generations don't bother learning about the actual details of how to do things without the machines/computers, so when something happens and you can't use them, the younger generations may not have the ability to maintain or fix things going forward because they've learned to just let the computers/machines do everything. As more and more baby boomers retire every day, we are losing experience and knowledge in many areas that the younger people aren't taking up. So say they do end up using AI/ML to eventually grade cards, after some time all the experienced human card graders who had been helping to maintain those automated gradings systems will have retired or passed. So what happens then if problems or issues occur requiring more human involvement, and there aren't really any experienced graders around?

And this is for everything when it comes to the younger generations, not just the grading of cards by machines. Just think how many times you've gone to a store and given a young cashier money, and if it wasn't for the register doing the calculation for them, they can't really figure out the proper change.

Last edited by BobC; 08-11-2021 at 04:52 PM.
Reply With Quote
  #33  
Old 08-11-2021, 04:53 PM
mrreality68's Avatar
mrreality68 mrreality68 is offline
Jeffrey Kuhr
Member
 
Join Date: Jul 2017
Location: Long Island, NY
Posts: 5,607
Default

I nominate Snowman to be the Official AI Representative of Net54baseball Forum!!!

Anyone want to 2nd the nomination
__________________
Thanks all

Jeff Kuhr

https://www.flickr.com/photos/144250058@N05/

Looking for
1920 Heading Home Ruth Cards
1933 Uncle Jacks Candy Babe Ruth Card
1921 Frederick Foto Ruth
Joe Jackson Cards 1916 Advertising Backs
1910 Old Mills Joe Jackson
1914 Boston Garter Joe Jackson
1915 Cracker Jack Joe Jackson
1911 Pinkerton Joe Jackson
Shoeless Joe Jackson Autograph
Reply With Quote
  #34  
Old 08-17-2021, 06:48 AM
hcv123 hcv123 is offline
Howard Chasser
Member
 
Join Date: Sep 2009
Location: NY
Posts: 3,422
Default WOW! Thank you snowman

I was guilty of loving and getting excited about the concept so much that I hadn't taken the time to understand it (nor would I have known even where to begin if I wanted to). Your explanation was an incredible, sad and necessary reality check. I suppose we'll see where things go from here.
__________________
I have been a Net 54 member since 2009 and have an Ebay store since 1998 https://www.ebay.com/usr/favorite_things

Cards for sale: https://www.flickr.com/photos/185900663@N07/albums

I am actively buying and selling vintage sports cards graded and raw. Feedback as a buyer: https://www.net54baseball.com/showthread.php?t=297262

I am accepting select private consignments of quality vintage cards (raw or graded) and collecting "want" lists for higher end ($1K+) vintage cards.
Reply With Quote
  #35  
Old 08-17-2021, 10:06 AM
Case12's Avatar
Case12 Case12 is offline
Casey
Member
 
Join Date: Nov 2018
Posts: 681
Default

Delete

Last edited by Case12; 08-17-2021 at 11:20 AM.
Reply With Quote
  #36  
Old 08-17-2021, 10:55 AM
Case12's Avatar
Case12 Case12 is offline
Casey
Member
 
Join Date: Nov 2018
Posts: 681
Default

Check out Malcom Gladwells book, 'Blink'. He wrote "The Tipping Point" also. He makes a good argument that scientific analysis is completely different from human analysis. And that even cursory expert human anaysis is almost always better and correct more often (key is expert).

Last edited by Case12; 08-17-2021 at 10:56 AM.
Reply With Quote
  #37  
Old 08-18-2021, 11:40 AM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

Quote:
Originally Posted by hcv123 View Post
I was guilty of loving and getting excited about the concept so much that I hadn't taken the time to understand it (nor would I have known even where to begin if I wanted to). Your explanation was an incredible, sad and necessary reality check. I suppose we'll see where things go from here.
I think most people are in the same boat, even the executives at Collectors Universe. They just see "AI" everywhere they look these days and see all the brilliant breakthroughs being made in other industries and assume it's coming to every "theater near you" for every industry, including theirs. It takes a lot of research and dedication to understand how it actually works though, and what sort of problems are and aren't good candidates for the application of AI & ML.


Quote:
Originally Posted by Case12 View Post
Check out Malcom Gladwells book, 'Blink'. He wrote "The Tipping Point" also. He makes a good argument that scientific analysis is completely different from human analysis. And that even cursory expert human anaysis is almost always better and correct more often (key is expert).
Both are excellent books. Blink in particular. I love the story about the tennis coach who can predict a fault serve the moment a player begins their swing, before the ball hits the net with alarming accuracy, but doesn't know why or how he is able to predict it. He just "knows" based on his experience.
Reply With Quote
  #38  
Old 08-18-2021, 04:17 PM
68Hawk 68Hawk is offline
Dan=iel Enri.ght
 
Join Date: Mar 2010
Posts: 370
Default

I would think that what is important in designing machine learning to detect anomalies such as re-coloring and trimming is the framing of the particular questions you are asking.

Wouldn't re-coloring involve the question: "Is all the ink laid down (non-autos) on the card of uniform droplet shape (or wet transfer or any other printing method) and 'age'? Even the saturation level (eg. height that the ink is sitting on the paper) would be different if you applied a modern ink to the stock that has a break in the surface to that which laid down at original printing.
Also It would be impossible to find ink of the same age as that used in a specific printing moment when the card sheets were first colored, if a 'red' spot luminesces (has different presentation qualities) differently to all the other red inks on the paper, you can be confident it was added at a different time.
No?

Re trimming, I'm imagining a similar question can be asked.
Whether that involves inspecting the side edge of the card's stock for a particular 'presentation' that a paper cutter leaves at its original operation, but which changes on any edge to receive a more recent cut...or some other important aspect which can universally be asked and requires a limited answer response for final judgement.

To the other aspects looking to be 'graded', I'm thinking also that it is the value of the question being asked to be machine learned that is of most importance.
I'm certainly not putting it past a hobby enthusiast or invested professional in the machine learning industry to be able to craft the kinds of questions that would lead to a satisfactory automated grading system.

Time frame? Who knows.

Last edited by 68Hawk; 08-18-2021 at 04:32 PM.
Reply With Quote
  #39  
Old 08-18-2021, 05:13 PM
Case12's Avatar
Case12 Case12 is offline
Casey
Member
 
Join Date: Nov 2018
Posts: 681
Default

I believe it can be done. It's not magic, but it can go beyond human learning capability very, very fast. But, by the time the database is confirmed and sensors designed, the ROI is really not there for this app. It sure would be fun to do it though.
Maybe we can get Elon Musk into collecting. :-)

Last edited by Case12; 08-18-2021 at 05:25 PM.
Reply With Quote
  #40  
Old 08-18-2021, 05:59 PM
doug.goodman doug.goodman is offline
Doug Goodman
Member
 
Join Date: Apr 2009
Location: On the road again...
Posts: 4,550
Default

.

Last edited by doug.goodman; 08-18-2021 at 06:16 PM.
Reply With Quote
  #41  
Old 08-18-2021, 06:15 PM
doug.goodman doug.goodman is offline
Doug Goodman
Member
 
Join Date: Apr 2009
Location: On the road again...
Posts: 4,550
Default

Quote:
Originally Posted by AndrewJerome View Post
Snowman,

Just wanted to thank you for the incredibly detailed breakdown. The information you provided is a fascinating read for us "card dorks."

Andrew
I just read your three posts and agree completely.

Thank you very much.
Reply With Quote
  #42  
Old 08-18-2021, 06:15 PM
doug.goodman doug.goodman is offline
Doug Goodman
Member
 
Join Date: Apr 2009
Location: On the road again...
Posts: 4,550
Default

Quote:
Originally Posted by mrreality68 View Post
I nominate Snowman to be the Official AI Representative of Net54baseball Forum!!!

Anyone want to 2nd the nomination
Second.
Reply With Quote
  #43  
Old 08-24-2021, 12:34 PM
T205 GB's Avatar
T205 GB T205 GB is offline
@ndrew woo.dfin
Member
 
Join Date: Dec 2012
Location: MN
Posts: 1,526
Default

Its not even close to working properly because this is what happens when your a Piece of Shit company and steal peoples ideas and work. I hope it all implodes leaving them with millions in damage control and a tainted reputation!
__________________
Andrew

Member since 2009
Reply With Quote
  #44  
Old 08-24-2021, 01:10 PM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

Quote:
Originally Posted by T205 GB View Post
Its not even close to working properly because this is what happens when your a Piece of Shit company and steal peoples ideas and work. I hope it all implodes leaving them with millions in damage control and a tainted reputation!
I feel like I'm missing something here lol. What's the story behind this post?
Reply With Quote
  #45  
Old 08-24-2021, 05:04 PM
T205 GB's Avatar
T205 GB T205 GB is offline
@ndrew woo.dfin
Member
 
Join Date: Dec 2012
Location: MN
Posts: 1,526
Default

Lets just say that it was to expensive and naïve of an endeavor for any grading company to pursue much less a commoner. Kinda like the Edison and Tesla story.

Quote:
Originally Posted by Snowman View Post
I feel like I'm missing something here lol. What's the story behind this post?


Great post above and welcome to the board. I hope you know I posted in response to the thread question and not your post. Had not read them till much later in the thread.
__________________
Andrew

Member since 2009

Last edited by T205 GB; 08-24-2021 at 05:21 PM.
Reply With Quote
  #46  
Old 09-22-2022, 07:47 PM
Snowman's Avatar
Snowman Snowman is offline
Travis
Tra,vis Tr,ail
 
Join Date: Jul 2021
Posts: 1,833
Default

I just wanted to post an update since I have recently looked into TAG grading and how they are attempting to automate the grading process with machine learning. They appear to have addressed what was probably my biggest concern of all, which is that high-resolution scans are insufficient for detecting surface flaws of cards even when those scans are being analyzed by "AI" (or machine learning).

Here is a link to their video about how their grading process/tech works:

https://youtu.be/5SPDCpYvDhQ

The key to what TAG is doing differently is not their "AI", but rather the imaging they are using to create scans that can show surface flaws. They are using something called photometric stereoscopic imaging which takes multiple images of a card using different light source angles with each image and then convolving them into one image that is then used by their machine learning algorithms to grade the cards. From the videos they've shared of how this works, I must say that I am impressed. You could definitely build some very useful grading algorithms with these images. Particularly with modern cards.

That said, many of the challenges I raised earlier still apply, and I think it will likely be a long time before they use this to grade vintage cards (if ever). The paper stock of vintage cards just has so much "noise" from a computer vision perspective that it would be extremely difficult (and VERY labor intensive) to build out these models even with photometric stereo imaging. They would still need to build a training data set for every disparate card type, and that is MUCH easier to do with ultra-modern cards where hundreds of thousands of copies are easily available. Building out a training data set for vintage cards would be much more difficult.

But this imaging technology does have the potential to be a game changer. As far as I'm aware, Genamint/PSA is not using this type of imaging. However, it is also perhaps worth pointing out that there is nothing TAG can do to prevent Genamint/PSA (or anyone else for that matter) from doing the same thing. Despite the numerous patents that they highlight on their website, one thing I've learned from working in this industry over the past decade-plus is that they absolutely cannot patent the use of machine learning algorithms for something like grading cards. You can't patent the application of machine learning for anything. It's like trying to patent a mathematical formula, and even if they somehow did receive a patent for something like this it would absolutely be thrown out if challenged. I was involved in a medical device startup several years back with a Harvard professor/heart surgeon and a team of patent attorneys who tried every trick they could to patent the use of AI/ML for all sorts of different purposes. It's simply not possible. Perhaps one of the patent attorneys around here could explain it better, but I promise you, they can't prevent another entity from doing the same thing, and they did not invent photometric stereo imaging.

That said, it looks like a pretty cool setup they have going. I'm hoping they do well and become a disrupter in the industry. There are a ton of problems I still foresee them encountering, and I could come up with a million questions I'd love to ask their team, like how they handle the fact that the grading scale we are all familiar with is very much non-linear with respect to differences in condition, but that's for another discussion.

If I have time, I may start a separate thread for TAG grading and revisit some of my points from earlier in this thread, but most of my concerns still apply. However, they do appear to have solved the riddle of getting surface flaws to show up in one single image, and that's a HUGE win!
__________________
If it's not perfectly centered, I probably don't want it.
Reply With Quote
  #47  
Old 09-25-2022, 07:11 PM
Leon's Avatar
Leon Leon is offline
Leon
peasant/forum owner
 
Join Date: Mar 2009
Location: near Dallas
Posts: 34,196
Default

I watched a few minutes of the video. It's interesting. They won't be doing vintage anytime soon though, so for me, it's a non-starter.

Quote:
Originally Posted by Snowman View Post
I just wanted to post an update since I have recently looked into TAG grading and how they are attempting to automate the grading process with machine learning. They appear to have addressed what was probably my biggest concern of all, which is that high-resolution scans are insufficient for detecting surface flaws of cards even when those scans are being analyzed by "AI" (or machine learning).

Here is a link to their video about how their grading process/tech works:

https://youtu.be/5SPDCpYvDhQ

The key to what TAG is doing differently is not their "AI", but rather the imaging they are using to create scans that can show surface flaws. They are using something called photometric stereoscopic imaging which takes multiple images of a card using different light source angles with each image and then convolving them into one image that is then used by their machine learning algorithms to grade the cards. From the videos they've shared of how this works, I must say that I am impressed. You could definitely build some very useful grading algorithms with these images. Particularly with modern cards.

That said, many of the challenges I raised earlier still apply, and I think it will likely be a long time before they use this to grade vintage cards (if ever). The paper stock of vintage cards just has so much "noise" from a computer vision perspective that it would be extremely difficult (and VERY labor intensive) to build out these models even with photometric stereo imaging. They would still need to build a training data set for every disparate card type, and that is MUCH easier to do with ultra-modern cards where hundreds of thousands of copies are easily available. Building out a training data set for vintage cards would be much more difficult.

But this imaging technology does have the potential to be a game changer. As far as I'm aware, Genamint/PSA is not using this type of imaging. However, it is also perhaps worth pointing out that there is nothing TAG can do to prevent Genamint/PSA (or anyone else for that matter) from doing the same thing. Despite the numerous patents that they highlight on their website, one thing I've learned from working in this industry over the past decade-plus is that they absolutely cannot patent the use of machine learning algorithms for something like grading cards. You can't patent the application of machine learning for anything. It's like trying to patent a mathematical formula, and even if they somehow did receive a patent for something like this it would absolutely be thrown out if challenged. I was involved in a medical device startup several years back with a Harvard professor/heart surgeon and a team of patent attorneys who tried every trick they could to patent the use of AI/ML for all sorts of different purposes. It's simply not possible. Perhaps one of the patent attorneys around here could explain it better, but I promise you, they can't prevent another entity from doing the same thing, and they did not invent photometric stereo imaging.

That said, it looks like a pretty cool setup they have going. I'm hoping they do well and become a disrupter in the industry. There are a ton of problems I still foresee them encountering, and I could come up with a million questions I'd love to ask their team, like how they handle the fact that the grading scale we are all familiar with is very much non-linear with respect to differences in condition, but that's for another discussion.

If I have time, I may start a separate thread for TAG grading and revisit some of my points from earlier in this thread, but most of my concerns still apply. However, they do appear to have solved the riddle of getting surface flaws to show up in one single image, and that's a HUGE win!
__________________
Leon Luckey
Reply With Quote
  #48  
Old 10-03-2022, 06:00 PM
Lorewalker's Avatar
Lorewalker Lorewalker is offline
Chase
Member
 
Join Date: May 2018
Location: Oakland, CA
Posts: 1,319
Default

Did not read it because, frankly IDGAF, but maybe someone here does.

https://twitter.com/Paul_Lesko/statu...1sNcjxshzTxcPw
__________________
( h @ $ e A n + l e y
Reply With Quote
Reply



Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is On

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Card Grading vs. Autograph Grading scooter729 Net54baseball Vintage (WWII & Older) Baseball Cards & New Member Introductions 9 08-20-2014 12:52 PM
BVG grading Vintagevault13 Net54baseball Vintage (WWII & Older) Baseball Cards & New Member Introductions 14 05-11-2014 01:51 PM
Mint Grading, or is it the grading of mints? brianp-beme Net54baseball Vintage (WWII & Older) Baseball Cards & New Member Introductions 2 10-30-2010 09:11 AM
Scarcity, Real, Artificial, and Imagined Archive Net54baseball Vintage (WWII & Older) Baseball Cards & New Member Introductions 34 09-23-2007 05:27 PM


All times are GMT -6. The time now is 05:05 AM.


ebay GSB