VeggieTales Scripting Reference

This is documentation for editing scripts. For general usage and macro information, see Jimbly's Macros

This reference is now included in the VeggieTales installer under the file name ScriptingReference.txt. This wiki page last updated from VeggieTales version 1.60.

General Philosophy

Most of the existing macros and the macroing interface are all built around a few primary goals:

• No keyboard interaction. While a macro is running, the user should be able to freely chat with their friends, navigate between guild tabs, etc.
• Image recognition over offsets. Where possible, read and scrape the screen to find the exact image we want to click on instead of hardcoding pixel offsets that might change based on resolution or other factors. The downside of this is that it is more susceptible to minor changes in ATITD causing the macros to work.
• Status and prompts. Friendly prompts about what values are desired are easy to add, and much better than telling someone to edit the script to handle the values they desire.

API

The functions prefixed with "ls" (short for "LUA script", which makes more sense in the C++ code, as opposed to the actual macros where *everything* is lua script...) generally operate on the VeggieTales window, so lsGetWindowSize() gets the width and height of the VeggieTales window.

The functions prefixed with "sr" (short for Screen Reader) generally operate on the ATITD window, so srGetWindowSize() gets the width and height of the ATITD window.

The functions with neither an sr or ls prefix are lua-script library functions.

• Note that you *MUST* call srReadScreen to update the screen before calling srFindImageInRange etc.

Function List

General

 void lsSetCaptureWindow()
 void lsSleep(int ms)

Sending Input

 void srClickMouse(int x, int y, bool bRightClick)
 void srClickMouseNoMove(int x, int y, bool bRightClick)
 void srKeyEvent(const char *s)
 void srLeftArrow()
 void srRightArrow()
 void srUpArrow()
 void srDownArrow()
 void srDrag(int x0, int y0, int x1, int y1, bool rightClick)
 void srSetMousePos(int x, int y)
 void srMouseDown(int x, int y, bool rightClick)
 void srMouseUp(int x, int y, bool rightClick)
 Vec2 srMousePos()

Screen Reading

 Vec2 srGetWindowSize()
 Vec2 srImageSize(const char *fn)
 void srMakeImage(const char *name, int x, int y, int w, int h)
 Vec2 srFindImage(const char *fn, int tol)
 void srShowImageDebug(const char *fn, float x, float y, float z, float scale)
 Vec2 srFindImageInRange(const char *fn, int x0, int y0, int w, int h, int tol)
 void srReadScreen()
 int srReadPixel(int x, int y)
 int srReadPixelFromBuffer(int x, int y)
 void srSaveLastReadScreen(const char *fn)
 void lsShowScreengrab(const char *_color)
 Vec2[] lsAnalyzePapyrus(int minsize, int maxsize, int debug, int relevant_radius)
 Vec2[] lsAnalyzeSilt(int minsize, int maxsize, int debug, int relevant_radius)
 [l,t,r,b] srGetWindowBorders(int x, int y)
 Vec2[] findAllImages(String imgName, int tol)

Text Parsing

 Region object is a table where region[0] = x0, region[1] = y0, region[2] = width, region[3] = height
 Parse object is a table where parse[0] = x0, parse[1] = y0, parse[2] = text
 region[] findAllTextRegions()
 parse findText(String s)
 void stripRegion(region r)
 parse[] parseRegion(region r)
 parse findTextInRegion(region r, String t)
 region[] findAllRegionsWithText(String t)
 parse waitForText(String t)
 void waitForNoText(String t)
 region waitForRegionWithText(String t)
 parse waitForTextInRegion(Region r, String t)
 void clickText(parse p)
 parse[] getChatText()
 parse[] getAllText()
 parse findTextInParse(parse[] p, String t)

I recommend not using the below functions unless you're modifying lua library functions.

 void srStripScreen()
 void srStripRegion(int x0, int y0, int w, int h)
 region srFindFirstTextRegion()
 region srFindNextTextRegion()
 parse[] srParseTextRegion(int x0, int y0, int w, int h)
 region srFindInvRegion()
 region srFindChatRegion()
 void srTrainTextReader(int x, int y, const char *c)

User Interface

 int lsScreenX
 int lsScreenY
 Vec2 lsGetWindowSize()
 Vec2 lsMouseClick(int x, int y, int w, int h, int button)
 bool lsMouseIsDown(int button)
 void lsPrintln(const char *s)
 bool lsShiftHeld()
 bool lsAltHeld()
 bool lsControlHeld()
 void lsPlaySound(const char *sound)
 int lsPrint(int x, int y, int z, float xsc, float ysc, const char *color, const char *str)
 void lsFontShadow(int on)
 int lsMessageBox(const char *title, const char *msg, int buttons)
 int lsPrintWrapped(int x, int y, int z, int w, float xsc, float ysc, const char *color, const char *str)
 void lsDisplaySystemSprite(int tile, int x, int y, int z, float w, float h, const char *color)
 bool lsButtonText(int x, int y, int z, int w, const char *color, const char *txt)
 bool lsCheckBox(int x, int y, int z, const char *color, const char *txt, bool value)
 int lsDropdown(const char *key, int x, int y, int z, int w, int val)
 void lsScrollAreaBegin(const char *key, int x, int y, int z, int w, int h)
 void lsScrollAreaEnd(int h)
 void lsSetCamera(float x0, float y0, float x1, float y1)
 void lsDoFrame()
 void lsTopmost(int value)
 {bool done, selection} lsEditBox(const char *key, int x, int y, int z, int w, int h, float xsc, float ysc, const char *color, const char *def)
 int lsGetTimer()

Specific reference

void srKeyEvent(const char *s)

Sends a character or sequence of characters to the captured window. Example:

 loadfile("luaScripts/screen_reader_common.inc")();
 function doit()
   askForWindow();
   lsSleep(250); -- wait for shift release to make it to the window
   srKeyEvent('username\tpassword');
   lsSleep(250); -- wait a moment before submitting so we can see it work, that's more fun!
   srKeyEvent('\n');
 end

This will send the string "username" followed by a tab (\t) and then the string "password" and then enter (\n).

Misc Notes

Explanation of tol (tolerance) value in srFindImage() and srFindImageInRange()

Me: Can you explain how the tolerance range works srFindImage("waterw.png", 1) . I've seen it range from 1 to 5000, how does the range work?

Jimbly: srFindImage looks for a block of pixels similar to the specified image, where no pixel has a summed squared difference larger than the tolerance. A tolerance of 0 or 1 would require an exact pixel for pixel match, whereas a larger tolerance would allow for mild variation (most often the background behind the text is different, since ATITD has a textured background behind windows).

Jimbly: An example difference is if the provided image has an RGB of 100, 150, 200, and the screen read had an RGB value of 110, 145, 180, the summed squared difference is 10*10 + 5*5 + 20*20 = 525

Jimbly: Default tolerance, if none is specified is 4500, which seemed to always find a match, and never get a false positive. I seem to remember using 5000 everywhere, but just saw in the code it uses 4500 if none is specified

Jimbly: I've found things that use a low tolerance are much more likely to break (most common being if, say, another line of text is added to a window, the tolerance of 1 will certainly not find what its looking for anymore, where as a higher tolerance will find it even if it's lower on the screen (and has slightly different background behind the text).

Summary: Its probably safer to start out leaving the tolerance blank (default of 4500) unless you have a specific need to find something precise down to the last pixel. ie use srFindImage("Harvest.png"); instead of srFindImage("Harvest.png", 1); If you have a need to narrow down a search to be more precise, then start adding numbers to the tol value.

lsDoFrame()

This draws the VeggieTales UI, any queued up lsPrint, buttons, etc. So, if you want the user to see anything, it needs to be called. Just drawing text over and over in a loop won't ever show up on screen unless lsDoFrame(); is called.

Text Recongition Notes

See Skyfeather's OCR Notes