MacroAI User Guide

1. Introduction

MacroAI is a Windows desktop macro automation tool. You can create automated workflows for mouse clicks, keyboard input, image recognition, OCR text recognition, screen change detection, and more through action recording, visual node editing, or Lua scripting.

Use cases: Office automation · Software testing · Game assistance · IT operations · RPA

2. Quick Start — Get Started in 5 Minutes

MacroAI organizes all content (nodes, scripts, images, recorded macros, etc.) into projects, so you must create a project first.

2.1 Create a New Project

Click menu Project → New (or press Ctrl+N):

1. The first time you use it, you'll be prompted to choose a working directory (where project files are stored)
2. Enter a project name
3. Set the design area — the effective range for your macro operations:
4. Full Screen: uses your current screen resolution
5. Custom Selection: drag to select a window or region

2.2 Interface Overview

The main window has 8 tabs (switch via Ctrl+1~Ctrl+8). Start with these:

The toolbar provides Run/Stop, Background/Foreground mode toggle, Click Visualization (🔴), Debug Mode (🐞), and more. The status bar at the bottom shows the run state.

2.3 Record & Play Back a Macro

Recording is the fastest way to create a macro — record first, then refine:

1. Switch to the Macro Manager tab (Ctrl+7), click + New Recording
2. A green border appears around the design area, with a 「■ Stop Recording」 button at the bottom-left
3. Perform the actions you want to automate (clicks, text input, etc.)
4. If an action goes outside the design area, the border flashes red and the action is not recorded
5. Click 「■ Stop Recording」 to finish
6. In the dialog that appears, click Save to Project
7. Select the recorded macro in the list and click ▶ Play to replay it

2.4 Enhance Your Workflow with the Node Editor

A recorded macro can only be replayed as-is. Want to add conditions, loops, image recognition, or other logic? Use the node editor:

1. Switch to the Node Editor tab (Ctrl+1)
2. In the left panel, click the Call Recorded Macro node to add it to the workflow
3. Double-click the node to open its configuration and select the macro you just recorded
4. Add other nodes before or after it (e.g., Wait, Find Image) to build smarter automation
5. Drag nodes to reorder them, or drag them into a Condition or Loop node to group them together

Changes in the node editor automatically generate script code, which you can view and edit in the code editor.

2.5 Run Your Project

Click ▶ Run (or press F5) to execute the current project's node workflow / script code. Click ■ Stop (or press Shift+F5) to terminate at any time.

Tip: Turn on 🐞 Debug Mode in the toolbar to see more detailed log information for troubleshooting.

3. Core Features

Image Recognition

Find a specific image on the screen and perform actions:

1. In the Image Manager tab (Ctrl+4), upload a template image or take a screenshot
2. In the node editor, add a Find Image node
3. Configure: select the template image, search region, match threshold (default 0.85), and match method
4. Enable Click to automatically click when found; set a Timeout to keep searching until the image appears
5. The result is accessible via variables: result.center_x, result.center_y, etc.

OCR Text Recognition

Recognize text on the screen:

• OCR Recognize: select a region and read the text content
• OCR Compare: check if specified text appears in a region; enable Click to auto-click when matched
• Configure the default recognition language in Settings (Ctrl+,)

Screen Change Detection

Wait for specific changes on the screen, useful for monitoring UI states:

• Wait for Screen Change: detect content changes within a specified region. Supports sensitivity adjustment, periodic change filtering (e.g., blinking buttons), and timeout settings
• Wait for Bar Change: track length changes in bars (health bars, progress bars). Supports auto-detection of horizontal/vertical orientation and scrollbar interference filtering

Mouse & Keyboard Simulation

• Mouse Click / Mouse Move / Mouse Drag: supports direct coordinates and variable references; you can box-select to get coordinates
• Key Press / Type Text: simulate keyboard input and shortcut combinations
• Coordinate fields accept: direct numbers, variable references (e.g., result.center_x), or click-to-pick

Flow Control

• Wait: insert a delay between actions (in milliseconds)
• Condition: executes different branches based on conditions (previous step succeeded/failed, custom expression); can hold other nodes inside
• Loop: repeats its children; supports breaking out of the loop; can hold other nodes inside
• Call Function: execute a reusable sub-workflow
• Call Recorded Macro: replay a recorded macro within the node workflow

Every node can have an execution condition (run on previous success, on previous failure, or a custom expression). Loop and Condition can hold other nodes inside them — just drag nodes in and out to group them together.

Variables & Data Passing

Define variables (name, type, default value) in the Variables tab (Ctrl+3). Insert variable references in node configuration via the ▼ menu:

Audio & Notifications

• Play Sound: system sounds, do~si beeps, custom audio files
• Play Music: background music in common formats; the floating panel shows playback controls (pause/resume/mute); sound effects automatically lower music volume
• Show Notification: display a notification message at a specific step during execution

4. Advanced Features

Functions & Custom Actions

Functions: package a group of nodes into a reusable sub-workflow with parameter support. Create and edit them in the Functions tab (Ctrl+5).

Custom Actions: bind a function to a one-click button on the floating panel. Configure them in the Custom Actions tab (Ctrl+6); the buttons appear on the floating panel when running.

Regions & Multi-Resolution Support

A region is a named rectangular area on the screen:

• Design Area: the reference coordinate space defined when creating the project
• Run Area: the actual screen region at runtime (can differ from the design area)
• Custom Regions: user-named areas that can be referenced in node configuration
• Coordinate Strategy: determines how coordinates are mapped from design area to run area

Coordinate Strategy

Switch in Project → Settings → Region Settings (Ctrl+,):

• Uniform Scaling (uniform) — games, full-screen apps: coordinates scale by the proportional size of design/run areas. During calibration, freely drag-select the area while maintaining the same aspect ratio as the design area
• Pan Adaptation (desktop) — browsers, Office windows: coordinates shift only by translation, no scaling. During calibration, only the position can be adjusted; width and height are fixed to the design area

Switching strategies changes how all existing coordinate points are interpreted. A warning is shown, and you may need to re-adjust existing coordinate points.

Calibrating the Run Area

When a script is created at one resolution but needs to run at another (or the window position changes), click Project → Settings → Region Settings → Calibrate to re-set the run area:

1. If the design area is Full Screen: the run area is automatically set to the current full screen — no manual selection needed
2. If the design area is Custom Selection:
3. Uniform mode: drag to select the run area, maintaining the same aspect ratio as the design area
4. Desktop mode: click or drag to specify the run area's position (top-left corner); width and height are fixed to the design area

After calibration, all coordinates are automatically recalculated according to the current coordinate strategy.

Node-Level Override

Each Find Image node can independently set its own coordinate strategy (Pro feature): - Follow Project (Default) — inherits the global coordinate strategy - Uniform Scaling — forces uniform scaling mode - Pan Adaptation — forces pan adaptation mode

Useful when the project uses one strategy globally but individual nodes need a different strategy due to interface characteristics.

Debugging Tools

• 🔴 Click Visualization: when enabled, a red dot appears at each click position (disappears after 0.5s) for verifying coordinates
• 🐞 Debug Mode: when enabled, logs show more detailed execution information
• Log Output node: write custom log messages at key steps

Settings

Click menu Project → Settings (Ctrl+,) to open the settings panel:

• Working Directory: default location for project files
• Interface Language: 中文 / English
• OCR Language: default language for text recognition
• Auto Save: automatically save changes to disk

5. Interface Reference

Floating Control Panel

Automatically appears when running a macro. Always on top and draggable:

System Tray

Closing the window minimizes it to the system tray. Double-click the tray icon to show the main window. Tray menu: Show Window / Run Current Macro / Stop / Exit.

Keyboard Shortcuts

6. FAQ

Why does my recorded macro always click at the wrong position?

Make sure you set the correct design area when creating the project. If you change monitors or resolution, use the Calibrate feature to re-set the run area.

Image search keeps failing?

• Lower the match threshold (default 0.85, try 0.7)
• Narrow the search region for better speed and accuracy
• Ensure the template image matches the actual runtime environment
• Try lowering the match threshold to find a balance between precision and tolerance

What's the difference between Background and Foreground mode?

• Background Mode (default): simulates actions in the background without interfering with normal use
• Foreground Mode: directly controls the mouse and keyboard, taking over control

If background mode doesn't work for certain actions, switch to foreground mode. Use 🔴 Click Visualization to verify coordinates.

How do I debug my workflow?

Turn on 🐞 Debug Mode in the toolbar for detailed logs, add Log Output nodes at key steps, and enable 🔴 Click Visualization to see where clicks land.

What is the relationship between the Code Editor and Node Editor?

Operations in the node editor automatically generate Lua script code. You can view and manually edit the code in the code editor — they stay in sync.

How do I activate Pro features?

All Pro features (Advanced Vision, Audio & Notifications, Custom Code) are currently free to use. Check the status in the About dialog.

7. Demo Projects

MacroAI comes with built-in demo projects to help you quickly understand the software's capabilities.

Included demo projects:

How to import:

• When setting the working directory for the first time: the system will ask if you want to import demo projects
• Manual import: click menu Help → Import Demo Projects to import demos into your current working directory at any time

After importing, use Project → Open to view and run the demo projects. You can freely modify their scripts and nodes as a starting point for learning and customization.

For detailed API reference, see menu Help → API Reference*. *