Bottom Line: Getting AppleScript to simulate clicks and keystrokes can be frustrating, but using Accessibility Inspector and the “UI Elements” command can make it easier.

Update 20140802: The most important part of the post starts down at the UI elements section below. Also, keep in mind that you have to tell System Events to tell application processUI elements… if you forget “process” it’s not going to work.

I generally think AppleScript is a lot of fun. It’s close enough to plain English that even beginners like me can read a script and understand a good amount of what it’s trying to accomplish, then use that as a framework to write their own custom scripts to automate myriad tasks on a Mac.

When you create a Script Editor document, select a scripting language in the navigation bar. Figure 5-2Setting the scripting language in a Script Editor document window. If you always use the same language, set it as the default in the General pane of Script Editor preferences.
Learn how to use Script Editor on your Mac to create tools, apps, and scripts that perform repetitive tasks, automate workflows, and more. Global Nav Open Menu Global Nav Close Menu. To explore the Script Editor User Guide, click Table of Contents at the top of the page, or enter a word or phrase in the search field. Script Editor User.

Applescript Editor 2.0 Download Mac Free

This focus allows it to deliver a suite of tools that make AppleScript development amazingly productive. You can use it to write and edit code, analyze target applications, debug scripts, and more. Script Debugger has been making AppleScript better for over 25 years! Script Debugger 8 continues the tradition with support for Apple M1 Macs. The AppleScript Toolbox can be installed in 3 different places. When AppleScript Toolbox is used by an application you can embed the plugin inside the application bundle's resource folder. The advantage is that the application will not only use the correct version of AppleScript Toolbox, but the osax will always travel with the application.

One of the handy things AppleScript can do is simulate mouse clicks and keyboard keystrokes through a process called UI Scripting, which uses the System Events app. I think Python might be able to accomplish this with Appscript, but other than that, I’d say AppleScript is one of the only ways to do this. One example of putting this to use is my UpdateiOSApps.scpt I wrote a few months ago, which uses some UI scripting to get iTunes to the iOS Apps screen (cmd 7), check for new apps (cmd r), and download the updates.

Now that last part is really the trick. You don’t want to tell it to just click a certain point on the screen, since if the screen layout changed for some reason it might just click the “delete everything and make my Mac explode” button or something. Besides, if you wanted to share your script, everyone with a different screen size or device would have to reinvent the wheel.

Instead, you have to navigate through tons of ambiguously named documents and windows and UI elements and find out what they want to be called and build those into your Applescript. One tool that helps out considerably is Apple’s Accessibility Inspector, which comes as part of Xcode (available for free in the Mac App Store).

I’m not inclined to give a comprehensive overview of how Accessibility Inspector works, but I will mention a few things. This will probably make more sense if you start it up as you read along.

So this is what Accessibility Inspecture looks like. Both the Hierarchy and Attributes panes are dynamic and will change as you move your mouse cursor over various parts of a given app. Like it says at the bottom, you can lock it (cmd F7) while hovering over a particular item that you’re interested (such as a button you want the script to click) to give you a static display of the hierarchy and attributes of that UI element.

For example, here’s the Download All Free Updates button in iTunes that I alluded to above…

…and here’s the Accessibility Inspector when locked on that element.

Notice the four triangles down bottom. These are useful for navigating to parents, children, and siblings of a given UI element. What this means is that you might be looking at a screen with a divider and a button on the right side of the divider. In such a case, it’s possible that the screen UI Element is set up as the “parent” of the divider UI element, which is the parent of the button “UI Element.” You’ll eventually be using these to build a command such as

However, one of the biggest problems is what to call each UI Element. As you can see in the Attributes, there are descriptions and roles and titles…. and while I have seen some scripts employing strategies like ‘tell the first UI element whose role is “AXLink” to…’ and others that just use the title (e.g. ‘tell “loading iTunes store” to…’), I have had really inconsistent results doing this. When they have none of these attributes, you really have no choice but to use a number (as in “tell window 1”). However, this can be problematic as well, as different view options (e.g. sidebars) may change the number of the element you want. Gah.

Even worse, you’ll find that what is suggested by Accessibility Inspector (AI) and what AppleScript Editor likes are often two different things. For example, it took me quite a bit of Googling to figure out that while AI tells me that an element has the role “AXSplitGroup” and description “split group,” Applescript Editor only understands when I refer to it as a “splitter group.” While this makes sense once you’re told, I certainly did not intuitively think that I should try this name when fed “split group” and “AXSplitGroup” by Accessibility Inspector.

Applescript Editor 2.0 Download Mac

I’m going to jump tracks a bit here to talk about why this matters. When talking to AppleScript, you have to specify a full hierarchy to get anything done. As I said before, you’ll be doing this through the System Events app, which “tells” the process (not the application — note that the process may have a different name, which you may be able to divine using Activity Monitor, which is preinstalled on your Mac), which “tells” its children UI Elements what to do. Also, some parts of UI Scripting can only work on what it can see, in the same way that you can’t click a button that you can’t see. So when we start UI Scripting, we’ll start our AppleScript with a few lines that get things set up by starting the conversation with System Events and bringing the application in question into focus.

Try this out. It should bring iTunes into focus. Now on some systems (esp. older ones like mine) the app might not quite make it into focus before the next line of AppleScript is run, which presents a problem. For this reason, I’ll often add a delay 1 to give the app a second to come fully into focus.

UI Elements

Applescript Editor For Windows

Here’s where I tell you the one tip that makes this post worthwhile. Well, I was glad to learn it, at least. It’s that you can give the simple command UI elements to UI Scriptable UI elements, and they will return a list of their elements — with names that work — to the Results window in Applescript Editor. For example, here is a screenshot of the script with the delay and “UI elements,” including the returned results.

If you’ll scroll up a bit to the screenshot of Accessibility Inspector while locked onto the “Download All Free Apps” button, you’ll see in the hierarchy (top) pane that our top-level UI element is application. We’ve taken care of this with the to tell process “iTunes”. The next level below that is AXWindow:AXStandardWindow … so we’re looking for a window. Sure enough, in the AppleScript Editor results pane, we see a “window” named “iTunes.” So then we can “tell” this UI element to list its UI element children. Here is how we would do that, including the results (as a comment below the script). As you can see, we have a lot more to sort through this time.

Looking at Accessibility Inspector, the next UI element we’re looking for is an “AXSplitGroup.” As I mentioned above, I had a heck of a time trying to figure out what AppleScript Editor wanted me to call “AXSplitGroup.” Sorting through the returned results in the bottom pane, there is only one item that catches my eye — splitter group 1 — and sure enoug