Shannon Larratt is Zentastic

Current Version: 2.10

Main improvements in this version

Note: This version has major upgrades. I have not yet had time to update the instructions but I will do so when I can. Please let me know if you find bugs as I haven’t had time to fully test everything.

• Full support of searching Backpage.com and Daype.com has been added.
• Optional audio prompts added on search/scan complete, and on selected fail messages.
• If program is being run for the first time, it offers to properly install it, and then creates and shells embedded install tool.
• About page now displays lifetime bytes downloaded, searches performed, and search results found (as of this version of course; not retroactive).
• Program now keeps track of recent searches, and if you right click on the “load” button, it offers them up on a “quick-load” pop-up menu.
• Total time spent searching and total running time of program are now recorded (and display in the “About” dialog).
• Added a few keyboard shortcuts: Ctrl+L = Load, Ctrl+S = Save, Ctrl+N = New, Ctrl+G = Go! (and Ctrl+Enter), Ctrl+H = Help, Ctrl+D = Diagnostics Tab, “ESC” keyboard shortcut to close most dialogs.
• If dialog is tall enough and there is room for them, the scan URLs become simultaneously visible rather than simply selectable.

Please read the full update list near the bottom of this entry to see what’s been improved or fixed or changed.

ZenCASH version 2.10 here (282k)

You can also download ZenCASH version 2.08 here (101k), if deal-breaker bugs are found in 2.10.

ZenCASH (the “Zentastic Craigslist Assisted Search Harvester”) is a tool with a simple interface for doing bulk searches of Craigslist and Kijiji. It allows you to search for multiple terms in multiple cities at the click of a button, and view these results in a simple format in your browser or any software capable of loading a CSV file. It should work in any version of Windows.

General Notes

To install the program, just copy it into a directory and run it. It will automatically create its setup files in that same directory.

The first time you run the program, it will not yet have a list of cities or sections. Unlike previous versions, in this version those lists are not hardcoded, and are downloaded from the servers directly. They will be saved in the files “CLcitylist.csv” and “CLsectlist.csv” respectively for Craigslist’s data, and “KJcitylist.csv” and “KJsectlist.csv” for Kijiji’s data. They are comma separated volumes and can be imported into other programs. If you ever need to update them, just delete them and they’ll rebuild the next time you run the program. Settings (and window size) are automatically saved on exit, but you can always delete “ZenCLSH.ini” to revert to the defaults.

When you start the software, it should load the last search you were working on. It’s quite simple to use. Just go to the search settings tab, select the cities and sections you’d like to search, and then type in your search terms. Click “Go!” to start the search. It’s all pretty self explanatory, but if you get lost, you can always click the question mark and then on the item you want help with for a brief pop-up, or you can read this page of instructions of course.

* * *

Initial Section/City Download

The first time you run the software, it will need to visit Kijiji and Craigslist and download a list of sections and cities from each. These results will be saved in the save directory as the program to the files “CLcitylist.csv”, “CLsectlist.csv”, “KJcitylist.csv”, and “KJsectlist.csv”, which are standard comma separated volumes (so you can use them in other programs, or edit them manually as well should you need to). You’ll only need to do this once, but if for some reason you want to reset these files and download them again, just delete them and the software will re-download them.

* * *

Search Results Tab

This page shows the results of your search — all the matching entries that have been found so far. In the search box, you can double click an entry to view it, in its site (Craigslist or Kijiji), in your browser. You can also click on the column headers (ie. “Date”, “Title”, etc.) to sort the results by that column (otherwise the most recent finds will be inserted at the top of the list). If you’re using the keyboard, you can press delete to delete the currently selected item or group of items if you have multiples selected. You can also press letters or numbers to quickly search for results that begin with that character. You can also select multiple rows/results for use with the different output modes (CSV and HTML). Below the list you have a number of buttons. “Clear” deletes all your results and gives you a clean slate. “Filter” opens the “search inside search” dialog that lets you fine-tune your results. “Output CSV” lets you save your results to a comma separated volume, suitable for loading into databases, spreadsheets, and other applications that support that data format. “Output HTML” saves your results to an HTML file and then views that file in your default browser. For each of these, the default is to output all your results, although if you have selected specific multiple results on the list manually, they will be outputted. Alternately, if there are new results that have just been added, the software will offer to output just those. Finally, the “Deep Scan” button starts a download of the full text of the ads in the results list, and “Go!” actually begins the search.

* * *

Search Inside Search (Filter) Dialog

Clicking the “Filter” button takes you to this dialog, which lets you do very fast searches of everything currently in your results list. If you have downloaded the full data (via deep scan or via RSS descriptions), then it will be included in the searches. Typing searches in the text box cause the list below to be instantly filtered. In addition to being able to type in search terms (separated by spaces, with phrases being enclosed in quotes), which are treated as “AND” searches, you can type in a search term as a negative by prefixing it with a “-” sign. That is, if you type “-blue”, then any result that includes the word “blue” in the title or description will not be displayed. As with the results list, you can double-click on an item to see it in your browser, and you can use the “Output CSV” and “View HTML” buttons to output the filtered results of the list to a filename or your browser.

* * *

Search Settings Tab

The search settings tab lets you decide what cities and sections you’ll be searching and what you’ll be searching them for. Sections are displayed on the right, and cities are displayed on the left, with a “Select All” and “Clear All” button for each. If you are searching only Kijiji or only Craigslist, then only one set of cities and one set of sections will be displayed, but if you are searching both sites, then, because they are slightly different, two sets of lists will be displayed. If you’ve chosen to do so in the software settings, when you click a city on one list, it will do its best to try and find the same city on the other site’s list and make its checkbox match (with sections doing the same). Search terms let you choose the terms you’ll be searching for, with multiple terms being separated by commas. These are actually separate queries. If you want to make a single search with multiple terms, don’t separate those terms with a comma. If you want to search for an exact phrase, enclose that phrase in quotes. You can click the “Go!” button to start searching (it’s the same button as on the search results tab). When you save a search, all these settings will be saved along with the results.

* * *

Software Settings Tab

This lets you change the settings of the software — these settings are loaded globally for the program when it starts versus being on a per-search basis. These settings are stored in the file “ZenCASH.ini” in your program directory, which you can delete if you’d like to reset it to its defaults. The settings available are:

• Search Craigslist – This tells the program that you would like to include Craigslist in your searches.
• Search Kijiji – This tells the program that you would like to include Kijiji in your searches.
• Automatically do a deep scan – This tells the program to automatically download the full text of an entry after downloading its title.
• Check zentastic server for updates – This checks this page for updates to the software and lets you know if there have been (it will not automatically update for you though).
• Automatically discard old results – This throws away search results that are more than two months old automatically, to keep them from cluttering up and overflowing your results box.
• Maximum number of search results – This is the maximum number of results that the program should allocate space for. If you start to overflow, you can increase this as needed (note that the change won’t be reflected until your next load).
• Maximum number of search threads – This specifies the maximum number of concurrent internet connections to use for searches. Opening too many is faster, but can result in connection problems depending on your network.
• Maximum number of deep scan threads – This specifies the maximum number of concurrent internet connections to use for ad downloads. Opening too many is faster, but can result in connection problems depending on your network.
• Fake browser user agent – This specifies the browser to pretend to be while downloading.
• Only search in titles of ads – This tells the software to only search in the titles of ads, rather than in their full text. This setting is currently Craigslist only.
• Only search ads with images – This tells the software to only search for ads that have pictures. This setting is currently Craigslist only.
• Attempt to automatically synchronize – When you have this turned on, when you click a Craigslist city or section, it will try and automatically select the Kijiji one of the same name, and vice-versa.
• Attempt to harvest full text from RSS – This tells the software that when downloading a Craigslist RSS feed to grab the text of the ads from it, rather than doing a deep scan. This is much faster than doing a deep scan, but it does not include as much data, and it will skip the pictures.

Note: the following have been added since the screenshot was made:

• Re-download RSS pre-loaded descriptions when deep scanning – This has the program download the ads even if we already have their contents from the RSS feed (which may be truncated or not include pictures, so doing this is recommended). Entries with no data in them will be given priority, so these re-downloads are only done when there are no other tasks.
• Include site in CSV output – This creates a site name column when outputting the CSV file that indicates whether the result is in Kijiji or in Craigslist.
• Include state code in CSV output – This attempts to include the state/province code (ie. NY, AL, TX, ON, QLD, NSW) for the USA, Canada, and Australia, as a separate column. Unknown states will be listed as “–”. There may be some errors as Craigslist sometimes lists border cities in both states.

The “cancel” button aborts your changes and reverts to your last saved settings, and the “apply” saves your settings and starts using them from now on. The “about” button in the bottom right pops open the “About ZenCASH” dialog that tells you a little about me and the software.

* * *

About Dialog

This is what you get when you click the “About” button. It’s about what you’d expect, right? If you’d like you can clikc the “Visit Website” button to visit this page.

* * *

Diagnostics Tab

The diagnostics tab lets you know what the software is currently doing. The top row tells you how many search results are currently in the results box, as well as how many of them were discovered in this search session. Bytes downloaded indicates how many bytes of data in searches and ad downloads we’ve done via Craigslist and Kijiji combined — note how quickly this number can climb! The next twelve rows let you know what that search slot (between 1 and 12 may be active for a given session, as defined in your settings) is doing. Searches will show what is being searched for and where, and scans will indicate the URL being downloaded — it will also indicate whether there is a connect problem (these usually happen if you are searching too aggressively and you overwhelm your connection). The icon on the left side tells you at a glance whether it’s accessing Kijiji or Criagslist by showing the site’s icon. You can select whether scans or searches are being displayed by clicking the button for either one in the bottom right of the tab. Just to the left of these buttons are two rows of twelve indicator lights each, with the top row being searches and the bottom being scans (downloads). A dark/unlit light indicates that connection is dormant. A bright green light shows that it’s downloading data, and a red light shows that there was a connection problem and it’s waiting to reset and try again.

* * *

HTML Template Tab

When you click the “View HTML” button on the results tab, the HTML file that contains the ads is defined by the template on this page. The top (“header and container”) text box is the overall HTML page that contains the entries, and the bottom (“item fragment”) text box is the HTML that is repeated once for each search result being included in the file. Each of these two files contains a number of “macros” which are replaced by live data. In the top box, they are:

• **NAME** – This inserts the name of the search.
• **TIMESTAMP** – This inserts the time and date of the creation of this file, in the format “HH:MM:SS MM-DD-YYYY”.
• **ITEM** – This indicates where the list of items are to be outtted should go.

And in the bottom box, they are:

• **COUNTER** – This is a numeric counter indicating the item number. It’s useful if you are calling relevant Javascripts for example.
• **ITEMTITLE** – This is the title of the item (ie. the title of the ad).
• **ITEMDATE** – This is the date that the ad was posted in the format “YYYY-MM-DD”.
• **ITEMCITY** – This is the name of the city that the ad was found in.
• **ITEMSECTION** – This is the name of the section that the ad was found in.
• **DESCRIPTION** – This is the description or text of the ad, either found in the RSS file or downloaded via a deep scan as dictated by your settings.
• **NEW** – This inserts the text “New” if the item is newly discovered in this search session.
• **ITEMLINK** – This inserts the online URL of the item; a linkback to Kijiji or Craigslist.

The “save” button saves your changes to disc and starts using them from now on, the “cancel” button aborts any changes you’ve made and reverts to your last saved version, and the “reset” button reverts to the software defaults. The template is saved in your settings, so if the settings file is deleted, it will also revert to the defaults.

* * *

Revision History

v2.10 – December 31, 2009 (Major upgrade!)

• Optional audio prompts added on search/scan complete, and on selected fail messages.
• Full support of searching Backpage.com and Daype.com has been added.
• City/section selector page now handles anywhere between 1 and 4 sites.
• Added a pair of labels next to the “clear all” buttons that indicates what site the treeview that the mouse is over is for.
• Clear All / Select All buttons now collapse/expand treeviews as makes sense, and handle the personals section in Backpage correctly
• Internal search load/save file format (.zen files) have been updated to be more adapable and easy to update in the future (old format was difficult to keep backward compatible).
• Embedded proper Microsoft-approved version code and file info via resource file.
• If program is being run for the first time, it offers to properly install it, and then creates and shells embedded install tool.
• Optimized internal string handling for speed.
• When searching Daype, it can be done sitewise via RSS, or normally on a per-section basis via their HTML interface.
• When resizing the dialog, if you come within 150 pixels on each dimension, it will automatically maximize.
• Minimize and maximize buttons added and enabled, and context-help button has been moved down slightly (to accommodate Windows “rules”).
• If software settings changes are cancelled by leaving the tab, a message box informs the user.
• About page now displays lifetime bytes downloaded, searches performed, and search results found (as of this version of course; not retroactive).
• Program now keeps track of recent searches, and if you right click on the “load” button, it offers them up on a “quick-load” pop-up menu.
• Progress bar along bottom has been replaced with a status bar (which includes a progress bar and other information.
• Statusbar displays the file URL being downloaded in its left info pane.
• Added “NEW” search button which is basically just the clear button combined with entering a new name.
• Deep scans of Daype extract the correct date from the entry (otherwise the search uses today’s date for Daype results).
• Added abort search button to the diagnostics page for convenience.
• Total time spent searching and total running time of program are now recorded (and display in the “About” dialog).
• Double clicking on the results list on blank lines no longer tries to view anything.
• Redrew the diagnostic icons to be less garish and match the look of the “help” icon/button.
• Added a few keyboard shortcuts: Ctrl+L = Load, Ctrl+S = Save, Ctrl+N = New, Ctrl+G = Go! (and Ctrl+Enter), Ctrl+H = Help, Ctrl+D = Diagnostics Tab, “ESC” keyboard shortcut to close most dialogs.
• Added a pretty radar scope ZenCASH icon animation on the About dialog.
• Added an “author photo” easter egg if you click my name in the About dialog, or press the letter ‘S’.
• Fixed bug where “New Search” dialog was not properly modal.
• If dialog is tall enough and there is room for them, the scan URLs become simultaneously visible rather than simply selectable.

v2.08 – December 8, 2009

• CLcitylist.csv now includes the state code to help people who use this file in their own projects.
• Most Kijiji searches should now respect “search in titles” and “picture required” options.
• Instance control (multiple concurrent copies) code added.
• State-in-output bugfix.
• Option to turn off “view this in browser” dialog.
• Links in About box are clickable.

v2.07 – December 3, 2009

• Added “include site in CSV” setting that adds a “Kijiji/Craigslist” output column in CSV (thanks Tom McC).
• Fixed missing line breaks issue in CSV file output.
• Fixed some bugs in the initial city/section downloader and added CL state downloader for USA/Canada/Australia.
• Added URI to State array loader in load settings function.
• Added “output state in CSV file” setting to settings page, and added it to the CSV output functions.
• Pressing a letter/number while in the results list searches for something starting with that letter/number.
• Pressing DEL while in the listview of results will delete the currently select item or items.

v2.06 – December 1, 2009

• Fixed bug in CSV output that was causing the saved message to display over and over.
• Fixed bug in output loops that was causing some searches to be omitted.

v2.05 – November 30, 2009

• Progress bar now works for deep scan (properly indicates what percent of remaining full scans are done).
• Fixed bug in which output CSV displays a success message on every single item in the file (thanks Tom McC).
• Modernized look and feel to Windows XP style buttons, shading, etc.
• Removed horizontal scrollbars from HTML text boxes.
• Clicking “deep scan” or “search” now activates the appropriate display mode in the diagnostics tab automatically.

v2.04 – November 27, 2009

• Program renamed to “ZenCASH – Zentastic Craigslist Assisted Search Harvester” and got a pretty new icon
• Software can now automatically initialize/discover Kijiji city and section arrays
• Software now displays a list of Kijiji cities and sections, and only displays KJ/CL lists if selected in settings
• Search can now search either/or Kijiji and Craigslist, and output and view functions support it
• New setting added: Attempt to auto-map between Kijiji and Craigslist on cities, as well as on sections
• Search start (GO!) button now exists on both the search results and the search settings pane (thanks Tom McC.)
• Diagnostics tab now shows number of results, number of new results, and bytes downloaded (excluding version check)
• Deals with expired ads a bit better, and automatically removes them from the results
• Fixed bug that caused window size to shrink in height a few pixels every load
• Filter (Search within Search Results) Window, with view and output ability (Thanks Tom McC.)
• About dialog takes focus properly
• HTML output look-and-feel now managed by user-modifiable templates
• Section/city csv files are no longer sometimes erroneously showomg as not found
• First load no longer requires restart (program runs normally after getting sections).
• Added “working” animation to download cities/sections tool
• Released EXE is now compressed with UPX v3.04 which significantly reduces the size (to about 25%)
• Optionally use description field from RSS feed to seed deep scan / full text (via new checkbox on settings page — note that this means the preview won’t download images, which may or may not be a problem).
• Bug that was not properly outputting selected rows has been fixed

v2.02 – November 20, 2009

• Slightly changed HTML output, including tagging the day’s downloads with “NEW”.
• When saving CSV files and HTML files, you can output everything, or just today’s new entries.
• Permissions-level bug in “view in browser” shell command should be fixed now [untested fix].
• Collisions in listview no longer happen.
• You can select multiple items from the results listview to output or view just those items.
• Offers to add itself to start menu.
• Added “About” dialog and moved the “visit website” button there.
• Can now specify that search should be of titles only, not body text.
• Can now specify that results should only include ads with pictures
• Clicking on column headers sorts list by that column
• Source code made public… I sure hope it’s not too embarrassing
• Search name can be specified on the command line — ie. “ZenCLSH2 searchname”

v2.00/2.01 – November 19, 2009

• Complete rewrite.
• 2.01 is a minor bugfix.

v1.04 – August 6, 2008

• Last stable version before rewrite. Click here to visit page for this version.

* * *

Feel free to contact me with questions or feature requests, or post them here in the comments. If you use the source code or have questions about it, I’d love to hear from you.