Tip for clicking Java Swing app controls - Robin+AHK

This tip is the offspring of my not being able to click on Java Swing controls directly with Robin (at least so far). I use AutoHotkey’s ‘ImageSearch’ command to find a screen clip of the ‘Quit’ button in a dead-simple and otherwise useless Java Swing application.

Steps:

  • Run Java with AutoHotkey, not a Robin action, so the script can advance (here, “fj6.ahk”, see below)
  • Run the image find-and-click AHK script (“fj7.ahk”, see below)

Robin code:

AutoHotkey script ‘fj6.ahk’:

Run, javaw %A_ScriptDir%\QuitbuttonEx.java

AutoHotkey script ‘fj7.ahk’:

If (WinExist("ahk_class SunAwtFrame")) {
    WinActivate, ahk_class SunAwtFrame
    WinShow, ahk_class SunAwtFrame
}
sleep 2000
CoordMode, Pixel, Screen
ImageSearch, FoundX, FoundY, 0,0,A_ScreenWidth,A_ScreenHeight, %A_ScriptDir%\quit.png
if (ErrorLevel = 2)
    MsgBox Could not conduct the search.
else if (ErrorLevel = 1)
    MsgBox Icon could not be found on the screen.
else
    MsgBox, , , The icon was found at %FoundX%x%FoundY%., 1
sleep 1000

CoordMode, Mouse, Screen
FoundX := FoundX+30
FoundY := FoundY+20
MouseMove, %FoundX%, %FoundY%

MouseGetPos, OutX, OutY

MsgBox, , , x = %OutX% - y = %OutY%, 1
MouseClick, Left

Java code for ‘QuitButtonEx.java’, if you’re interested:

import javax.swing.GroupLayout;
import javax.swing.JButton;
import javax.swing.JComponent;
import javax.swing.JFrame;
import java.awt.EventQueue;

public class QuitButtonEx extends JFrame {

    public QuitButtonEx() {

        initUI();
    }

    private void initUI() {

        var quitButton = new JButton("Quit");

        quitButton.addActionListener((event) -> System.exit(0));

        createLayout(quitButton);

        setTitle("Quit button");
        setSize(300, 200);
        setLocationRelativeTo(null);
        setDefaultCloseOperation(EXIT_ON_CLOSE);
    }

    private void createLayout(JComponent... arg) {

        var pane = getContentPane();
        var gl = new GroupLayout(pane);
        pane.setLayout(gl);

        gl.setAutoCreateContainerGaps(true);

        gl.setHorizontalGroup(gl.createSequentialGroup()
                .addComponent(arg[0])
        );

        gl.setVerticalGroup(gl.createSequentialGroup()
                .addComponent(arg[0])
        );
    }

    public static void main(String[] args) {

        EventQueue.invokeLater(() -> {

            var ex = new QuitButtonEx();
            ex.setVisible(true);
        });
    }
}

The quit button (YMMV with this one, for sure). My display is scaled to 125%, so you’ll probably have to create your own. But it should look like this (note the smaller rectangle around the word ‘Quit’ - it needs to be there).

quit

And an animated GIF of the script in action (older version of ‘fj7.ahk’, new one has less delay).

Working

1 Like