I have some buttons that look really similar. I need code that clicks specific ones

https://imgur.com/a/5XsZo44

as you can see the only difference is the 1/6, 2/6, 3/6, ...

I tried OCR, it can't read it. I tried every kind of preprocessing imaginable. I don't see this working

I tried grayscaling these crops, grayscaling a screenshot, then template matching. This worked somewhat but it's not really accurate. It would find matches for 1/6 when it was on 4/6 etc. A problem is that the buttons slowly fill with a darker purple the longer the turn goes on, and there is no getting around that. Also at times these buttons will change text and say something like 'WAITING...' but my method still sometimes identifies it as a match and clicks it anyways, which is not ideal


Here's my code with the template matching
https://codeshare.io/N3D404
I haven't tried messing around with the threshold much but if anyone has recommendations for that I'd love to hear it

If you read through my code you'll see in the main loop it also looks for a PLAY and CONFIRM button; this works as intended because those two buttons are very distinct

I don't think I can do something like pixel-wise MSE because the buttons fill with a darker purple until the turn ends - which really messes up a lot of my terrible ideas

I've been playing with python for less than a week now, if anyone has any recommendations or methods to look into I would very much appreciate it

Have you tried to do Dynamic Thresholding, or even Dynamic Region of Interest? To solve some of the issues.

I do need to play around with some adaptive thresholding I think. The adaptive roi will be very useful later in this project

Could you share me the codeshare or repo? Above link isn’t working currently. You could always over-engineer with CNN 😂

i'm about to that point tbh
pm'd, thanks