Tesseract OCR Finetuning

This repository contains scripts and resources for finetuning Tesseract OCR on custom fonts and datasets.

📦 Features

• Train and finetune Tesseract OCR models.
• Support for custom fonts (e.g., LEGO Hattings).
• Evaluation metrics: CER, WER, confusion matrix.
• Integration with MLflow for experiment tracking.

---

🚀 Getting Started

1. Clone the Repository

git clone https://github.com/LEGO/Image-test.git

2. Create a virtual environment

Using WSL/Ubuntu OS

python3 -m venv venv
source venv/bin/activate

Using Windows

python3 -m venv venv
source venv\Scripts\activate

3. Install dependencies

pip install -r requirements.txt

🔧 Setup Tesseract training tools

For Windows

Download tesseract for Windows from this link

-Tesseract for Windows

For WSL/Ubuntu OS

Install tesseract from the command line

sudo apt-get install tesseract-ocr tesseract-ocr-dev libleptonica-dev

📂 Directory Structure

tesseract-ocr-finetuning/
│
├── data/                # LSTM training files (archive) and training data
├── fonts/               # Custom fonts for text2image
├── other_programs/      # Storing the old methods and miscellaneous programs
├── src/                 # Training and evaluation scripts
|   ├──JSON_files/       # Store paths of input and output directories in Windows  
|   ├──plots/            # Store heatmaps of data when running the tesseract test model
|   ├──shell_scripts/    # Execute these scripts to generate synthetic images and finetune
├── models/              # Output traineddata files
└── README.md

Add finetuned languages in your tesseract

From the models folder are the models that can be used for testing. Add these models in youe tesstrain folder in Windows.

For Linux, you can write a command

sudo mv models/lang.traineddata /usr/share/tesseract-ocr/5/tessdata/lang.traineddata

Add the font in the fonts folder and create a directory in your Linux system

mkdir -p ~/fonts
cp fonts/fontname.ttf ~/fonts/

You can modify the shell script in the shell_scripts folder by running legacytraining.sh to generate fine-tuned models if there are any new fonts.

To run the shell script to fine-tune Tesseract using the Legacy method, use the following commands

chmod +x <FILENAME>
./<FILENAME>

Setup Mlflow

Important

The current Mlflow workspace works under WSL/Ubuntu OS system. It is NOT configured for Windows.

Once you have finished the setup and have configured the paths of the input/output in a JSON file as shown:

{
    "stl_files_folder" : "/mnt/c/path_to_input_data",
    "output_folder_filled" : "/mnt/c/Users/path_to/447_Output-testvtk-edge",
    "stl_files_folder_test" : "/mnt/c/Users/test",
    "output_folder_filled_test" : "/mnt/c/Users/test_outputs/447_Output-testvtk-edge"
}

In the WSL terminal type start an Mlflow instance before running the programs. You can insert the path in convert_to_wsl.py to make the paths readable when running the program in WSL.

mlflow ui

Important

You need to have one terminal running the MLFLOW application, hence, you need to open a new WSL terminal to run the programs.

Running the programs

Once you have the Mlflow running in the background, you can run one trial or a set of 20 trials.

🧠 Running a single trial

To run a single trial, go to tesseract_train_pipeline.py and edit the params_vtk dictionary as seen here.

params_vtk = {
    "stl_files_folder" : data["stl_files_folder_KIWI"],
    "output_folder_filled" : data["output_folder_filled_KIWI"],
    "lang" : "Hattingam_legacyV2_text2img",
    "type" : "KIWI"
}

Edit the parameters of your choice and run the program. This logs in the parameters, the train and validation accuracy of the model, the individual character accuracy and a confusion matrix table (predicted text vs actual text, for example, '0' being classified as 'O').

This also plots the CER (Character Error Rate) of every individual character which will be logged in the "Artifacts" section in the "plots" folder.

More information

For more information on how to run the programs, see the Readme.md in src