Mini-Guide: Robin with AHK for finding pixel color

This post is in response to @supriya.shelke’s question regarding color detection. Even if it’s not presently possible to detect changes in pixel color directly in Robin, simple AutoHotkey scripts can help.

Goal: Detect whether we’re on the Robin forum page or the Robin docs page, using only the coordinates of the the “mascot” at the top of the web page. Do NOT use the title or any text to make this determination - find the RGB pixels of the “birdie” (0xEE0033) in the correct position.

Tools: Robin and AutoHotkey

AutoHotkey has a command called PixelSearch.

I suggest you review the documentation at the link given, but if that’s “TL;DR”:

  • OutputVarX is the ‘X’ coordinate of the found pixel
  • OutputVarY is the ‘Y’ coordinate of the found pixel
  • ‘X1’, ‘Y1’ are the upper left coordinates of the search area
  • ‘X2’, ‘Y2’ are the lower right coordinates of the search area
  • ‘ColorID’ is the hex representation of the pixel color - N.B. ‘BGR’ unless ‘RGB’ is specified!
  • ‘Variation’ is from 0 to 255 - 0 is the exact color, 255 finds all colors
  • ‘Mode’ = Fast, RGB, or Fast RGB

Here’s an AHK script that will search (on my system anyway, 125% scaling) for the “birdie” in two different places - one for where it is on the Docs page, one for where it is on the Forum page.

Different message boxes display which page was found, based on pixel search.

It’s for demo purposes only, and has no real error checking. You’ll likely have to adjust the pixel color and pixel coordinates based on your system. There are many, many tools available to grab the color of a screen pixel, the finding of which I will leave to the reader as another exercise. :grinning:

#NoEnv  
SendMode Input  
SetWorkingDir %A_ScriptDir%  
CoordMode, Pixel, Screen
IfWinExist, ahk_exe firefox.exe 
{
;msgbox Firefox exists
PixelSearch, OutX, OutY, 145, 145, 190, 190, 0xEE0033, 0, Fast RGB
;msgbox X - %OutX% : Y - %OutY%
If (!(!OutX || !Outy) && (ErrorLevel == 0)) {
	msgbox, , Where am I    , On Docs page, 1
	OutX := ""
	OutY := ""
}

PixelSearch, OutX2, OutY2, 260, 145, 290, 190, 0xEE0033, 0, Fast RGB
;msgbox X - %OutX2% : Y - %OutY2%
If (!(!OutX2 || !OutY2) && (ErrorLevel == 0)) {
	msgbox, , Where am I    , On Forum page, 1
	OutX2 := ""
	OutY2 := ""
}
OutX := ""
OutY := ""
OutX2 := ""
OutY2 := ""
return
}

ExitApp

Here’s a simple Robin script that launches Firefox, navigates first to the forum page, checks for the “birdie”, then navigates to the Docs page and does the same.

(Note: AutoHotkey can pass variables back to Robin. The results of the search are sent to standard error, but to get this information back to Robin you have to do more coding - something you’ll in all likelihood want to do. That’s beyond the scope of this exercise, but it is not difficult. Here, only AHK message boxes are used, as P.O.C.)

Code:

webautomation.LaunchFirefox \
    Url:  "https://forum.robin-language.org" \
    WindowState:WebAutomation.BrowserWindowState.Maximized \
    ClearCache:False \
    ClearCookies:False \
    BrowserInstance=> Browser
    
wait 7

System.RunDOSCommand \
    DOSCommandOrApplication: "AutoHotkey.exe C:\work\RobinTests\rc3.ahk" \
    WorkingDirectory:'' \
    StandardOutput=> StandardOutput \
    StandardError=> StandardError \
    ExitCode=> ExitCode
    
WebAutomation.GoToWebPage \
    BrowserInstance:  Browser \
    Url: "https://docs.robin-language.org/robin-the-rpa-language/"
    
wait 2

System.RunDOSCommand \
    DOSCommandOrApplication: "AutoHotkey.exe C:\work\RobinTests\rc3.ahk" \
    WorkingDirectory:'' \
    StandardOutput=> StandardOutput \
    StandardError=> StandardError \
    ExitCode=> ExitCode
    

WebAutomation.CloseWebBrowser BrowserInstance: Browser

Animated GIF:

Regards,
burque505

3 Likes