Sudoku Automation using ML algorithms

In this project, we develop sudoku (数独/すうどく) automation system to auto-solve sudoku puzzle from here. First things first, we grab the screenshot image and track every positions of digits and manually labels every digits by using OpenCV. Then, we implement KNearest-Neighbors-Optical-Character-Recognizer (KNN-OCR) to detect and classify all sudoku digit. Now, the 9x9 sudoku board is ready in our programe. We apply Backtracking to solve the puzzle. Finally, we simulate keyboard actions by pyautogui [Codes]

Normally, more screenshots are needed to train our KNN-OCR, so manully take few screenshot of sudoku and create "sudoku_train.png" by combining those. To skip this step, we provide you few sample screenshots under data/.

For data collection, run digit_recongizer.py. Please input corresponding number for each red rectangle of pop up window.

Now, training data and labels are all ready. ("sudoku_data.data" & "sudoku_labels")

Next, train KNN model by running knn_train.py. To skip this step, you can also use our trained model, "sudoku_digit_model.xml"

Now, go to sudoku.com. And run main.py.

IMPORTANTLY, mouse-click the digit on top-left corner of sudoku puzzle.

The programe will automically take 4 steps:

Screenshot sudoku;

Recongize digit and generate sudoku board;

Solve by Backtracking;

Simulate keyboard actions.

You may need to adjust WINDOW_X, WINDOW_Y, WINDOW_WIDTH and WINDOW_HEIGHT in main.py in order to take proper screenshot from your screen.

References

Python Sudoku Solver Tutorial with Backtracking [1]

Sudoku Automation [2]

Optical Character Recognition (OCR) with KNN classifier [3]

See other projects