I've started using ChatGPT for my work on BritBase. It has been a revelation, reminding me of the first time I started using the internet back in the 1990s.
BritBase is around 30 years old. One reason I started it was because I had noticed how inaccurate some of the data was on commercial chess databases, particularly the identity of players. For example, I had noticed that ChessBase seemed to think the Belgian grandmaster Alberic O'Kelly had played in the British Championship. He hadn't: it was his British namesake Rory O'Kelly. It struck me immediately that this sort of error could at least be lessened if there was someone from this windswept archipelago on Europe's west coast who was more familiar with its resident chess players and could act as curator of the games played in our vicinity. So I immediately appointed myself to the job and BritBase was born.
That's not to criticise the commercial database company who created the database. They had a software product to market and needed a database to accompany it. In fact, in the 1990s they did a remarkable job of inputting millions of games from all over the world, from a myriad of tournament bulletins and magazines. However, the necessity for speed in getting the product to market meant that inputters and editors weren't necessarily familiar with events and players in our part of the world. It also meant that the quality of the scores was less than ideal.
BritBase started life as a repository of game scores and nothing much else, but it has grown into something more ambitious in terms of tournament and match records, featuring tournament reports, biographical material, photos, etc. Although the technical side is just me and my increasingly aging web skills, it includes many valuable contributions from other people who have been kind enough to collaborate with me and lend support. Without them I would have given up years ago.
USING AI TO CREATE PGN FILES FROM SCANNED IMAGES
But I'll try and get to the point (like many a septuagenarian I am prone to lapse into memories and nostalgia!). The two most time-consuming areas of my self-imposed function are (a) inputting large numbers of games and (b) preparing big crosstables. Recently it struck me that I should see if AI could help to share the load. The short answer is, it can - big time. I thought it might be interesting to share my first thoughts about this, having only dipped my feet into the AI water.
First question: can AI be used to input games? Yes, it can but it requires good hardware and software as well as some IT savvy.
The primary difficulty is the same as it would have been 30 years ago: OCR (optical character recognition). Some years ago I tried to use software to scan material from printed sources and use OCR to interpret but soon ran into a snag. The software I was using, ABBYY FineReader 12, which I have to say is excellent for most purposes for which I use it, couldn't be 'taught' to interpret chess figurine notation. In theory it can do this but in practice it simply doesn't work well enough to be worth the bother. (Despite this drawback, I strongly recommend this reliable and useful software.)
OCR
The other problem with OCR is its (lack of) accuracy. To be fair, its accuracy percentage is actually very high but it can struggle to differentiate between certain printed characters, notoriously 'e' and 'c'. Even when scanning printed sources which pre-date the use of figurines there can be problems. Here's a simple fictitious example...
White to play. Let's suppose that the OCR software generates the move sequence 1 Ke2 Re7+ 2 Kd3 and the game continues. A chess player of almost any standard manually inputting the game would have spotted the problem immediately and realised that move 1 was probably 1 Kc2, not 1 Ke2, leaving a whole rook en prise, but OCR could easily make this mistake unless it has been carefully programmed to flag or correct such egregious errors. I have yet to find a good way to build a blunder-checking facility into the AI-driven process.
IT'S NOT ALWAYS OCR'S FAULT...
Having said which, such terrible moves do show up in printed bulletins. In only the second game I processed via AI to create a PGN file I came across this real-life example...
Here Black played 1...Rd6?? and the game concluded 2 Rb8+ Bc8 31 Rxc8+ Rd8 32 Rxd8 mate. I was suspicious and checked the printed page but found it really was 1...Rd6. OCR had not made a mistake. Those moves did get played (or, at least, were recorded as played in the bulletin). But in mitigation it was an Under 8 tournament!
READING (OR NOT READING) FIGURINE NOTATION
My second attempt to get ChatGPT to generate a PGN file from a series of scanned images was not immediately successful. My electronic assistant and I concluded that the resolution of the scan was not high enough and tried again at 600dpi. That worked better though AI needed a bit of help. It generated a file showing the figurines were still a problem. I decided to input one game manually, which conveniently had all the requisite figurine symbols in it, as an example for AI to use. It professed itself very happy with that approach and started work on processing the 205 games in the bulletin. It is still working on them as I write this... one hour and counting. But inputting 205 games manually would have taken me in excess of 10 hours, so it is still well in credit.
HOW GOOD IS YOUR SCANNER?
The previous paragraph highlights another problem: the quality of the scan. Or indeed the scanner. In my case, scanners, plural. I have a Canon R30 document scanner and a much older but still reliable Canon 9000F Mk II flat-bed A4 scanner. For multi-page documents such as tournament bulletins, the document scanner is a must, though it sometimes means having to unbind the bulletin in order to feed sheets separately. It scans at 600dpi which I'm beginning to think is essential for scans that are to be processed by AI.
CREATING AND REFORMATTING CROSSTABLES
The second area of work I intend to process via AI is the production of crosstables. So far ChatGPT has done a great job with these, processing scans from 1990s tournament bulletins, some of which have very small print, spread over two or more pages, and turning them into the standard BritBase crosstable format. It barely requires telling me how to do this in detail as it accesses BritBase pages directly and simply copies the format it finds there. Really very simple. It is very good at validating the data and checking player totals, etc.
So far, so good! Here's a final piece of advice for anyone thinking of doing something similar: before even thinking about using AI to do all these good things, buy a good quality scanner - and learn how to use it.
