CustomSearch CMP Tutorial

CustomSearch provides a powerful way to search for (and optionally replace) strings inside all major MODX objects on your site.

This is the full documentation page for CustomSearch. To see screenshots, and reasons why you might want it, go to Why Choose CustomSearch?

(Jump to System Settings Table.)

Click on the "Buy Now" link below to see the various licensing options for CustomSearch.


Purchase CustomSearch

Features

  • Extremely fast
  • Fully Compatible with MODX 3 and PHP 8
  • Search in almost all fields of Resources, Chunks, Templates, Snippets, Plugins, TVs, Users, and User Profiles.
  • Case-sensitive or case-insensitive searches
  • Case-sensitive or case-insensitive, site-wide search and replace
  • Regular expression searches!
  • Regular expression search and replace!
  • Shows found search terms in context
  • Links to edit objects found in search
  • Whole-word searches (and replacements)
  • Option to remember checkbox and max results settings
  • Dry-run option (with preview) for all replace operations

Version 1.3.0 Update Notes

  • Fixed security issue
  • Fixed issue with diacritics (non-ASCII characters)
  • Fixed issue with mbString handling
  • Improved Unit, Integration, and Acceptance tests
  • Better handling of 'property' field searches
  • Updated docs
  • Improved error handling
  • Refactored code example

Installing CustomSearch

Go to the secure server at Gumroad. Purchase and download the CustomSearch transport .zip file. It's not necessary to create an account at Gumroad, but if you do, you'll be able to re-download the package, or upgrade it to a new version, at any time in the future.

Once you've downloaded the .zip file, upload it to the core/packages directory at your site.

Note for Mac Safari users: Safari may automatically unzip the downloaded file. If you try zipping it back up, the re-zipped file may not be named properly. It has to end in transport.zip in order for Package Manager to find it.

See this page for information on how to prevent the file from being unzipped.

Also, at one time you could prevent the unzipping by holding the Alt key when you click on the download link. I'm not sure if that still works.

Once you've uploaded the .zip file, in the MODX Manager at your site, go to Extras->Installer on the Top Menu and click on the down arrow next to "Download Extras" and select "Search Locally for Packages." Click on "Yes" in the popup window that appears.

Find the CustomSearch package in the grid and click on the "Install" button.

Upgrading CustomSearch

When a new version of CustomSearch is available, you should receive an email with a link to it. Just follow the link, download the new .zip file, and follow the instructions above for installation. If you have created an account at Gumroad, you can also log in there and check for new versions of the package.

Usage

Once CustomSearch is installed, you should see it as a choice in the Manager's Top Menu. If not, reload the page. When you click on the "Custom Search" menu option, you'll see the CustomSearch form, with a list of MODX objects to search.

Put your search term in the "Search For" field of the form. Under each object you want to search, select the fields you want searched. Objects with no fields selected will be ignored in the search. See later section on Search and Replace.

Search Options

Below the search box are the options for the search:

  • Replace — Perform a search and replace operation
  • Dry run — Show search and replace results without making changes
  • Case-sensitive — Make the search case-sensitive
  • Whole words — Find only whole words
  • Regular Expression — Perform a regular expression search (and optionally replace)
  • Max Results per object — Limit results for each object searched (0 for no limit)

TVs

TVs are a special case in CustomSearch. It looks for TV values for resources. If you need to find the TV object itself by name or ID (for example, to change its default value), use the Manager's built-in search.

You should only select one of the three field options for TVs in CustomSearch (name, value, or id).

When you search for a TV by value, CustomSearch will actually look for Resources that have that value for the TV. The link provided for any match will be for the Resource, so that you can edit the value of the TV for that resource. This will only show Resource TV values that are not left at the TV's default value.

When you search for a TV by Id or name, CustomSearch will look for resources that have a value set for that TV. Again, the links will be to the resources. In the results, you'll see the name and ID of the Resource on the left and the name of the TV on the right. If you click on the link, you'll be editing the resource that has the selected TV. Below the link, you'll see the value of the TV for that resource (if any). Important: this will not show TVs set to their default value, since they don't appear in the TV value table in the database.

Searching for TV's is usually done by itself, but occasionally, you may want to include the TV value field in general searches along with other object fields.

Replace

When you check the "Replace" box, an input field for the replacement string will appear. All instances of the search term will be replaced by the replacement term (subject to the options you select). Regular expression search and replace is also an option. Be careful with search and replace — many search-and-replace operations are not reversible.

Always do a dry-run before executing a real search and replace operation to make sure the process does what you want. For major, site-wide search-and-replace operations, it's not a bad idea to back up the database first.

Regular Expressions

Regular expressions provide a very powerful way to search for (and optionally, replace) strings in MODX objects at your site. Using them to search is harmless, but a badly planned replace operation can trash your site. When planning regular expression search-and-replace operations you should always do a dry-run first.

Note that searching for MODX tags containing other MODX tags, especially ones containing output modifiers, can be extremely difficult (and sometimes impossible). Remember that you can find all tags of a certain type with a regular expression, but you may not be seeing all of the tag. For example: 

[[+login.update_success:if=`[[+login.update_success]]`:is=`1`:then=`[[%login.profile_updated? &namespace=`login` &topic=`updateprofile`]]`]]

It's almost impossible to capture this entire tag unless it is the only placeholder tag on the page. Search-and-replace operations with tags are usually safe when changing the name in the tag (e.g., placeholder name, chunk name, snippet name, etc.), but it's dangerous to try to change the entire content of a tag. Examine the "Dry Run" output carefully.

To search for all tags for a specific snippet, for example, you can do a regular expression search for this pattern to find all snippet tags called cached or uncached:

\[\[!?NewsPublisher

For all chunks with a given name, you can do this:

\[\[!?\$ChunkName

For all placeholders (may not show all of placeholders with output modifiers):

\[\[\+[^+\n$]+?]]

For all placeholders with a given name or prefix:

\[\[\+prefix[^+\n$]+?]]

When developing a regular expression, the Rubular Regular Expression Tester can save you a lot of time and trouble, though see the note below about the use of the ^ symbol.

CustomSearch (CS) uses MySQL's extremely fast REGEXP operator to perform regular expression searches. That means that your regular expressions should *not* include a delimiter at each end. CustomSearch will add curly braces at each end ({}) as delimiters. These delimiters were selected because you'd seldom want to search for them. If you use curly braces in your search pattern, you'll need to escape them with a backslash, like this:
\} else

The MySQL REGEX process uses the POSIX regex engine, rather than the more familiar PCRE one, so some of the expressions you're used to may not work (such as \d and non-greedy expressions). There's a good cheatsheet showing the differences between regex engines here.

Regular expression replacements are not done with MySQL, however, they're done individually with preg_replace(). This gets around the fact the MySQL doesn't recognize non-capturing groups.

The use of the ^ and $ symbols in CustomSearch (CS) can be a little confusing. In MySQL, the ^ symbol always refers to the beginning of the whole field, and patterns that begin with it will only search up to the first newline in the field being searched. That means the pattern ^Hello will only find "Hello" if it's the very first word of the field being searched. It will not find "Hello" when it is at the beginning of other lines in the field. To find it elsewhere, you can leave out the ^. If you want to find "Hello" as the first word of any line, do this: (^|\n)Hello.

Diacritics

Diacritics are accented, non-ASCII characters such as the acute accent (á), grave accent (à), and umlaut (ä). As of Version 1.3.0, you can now search for words with diacritic characters in CustomSearch. You do not need to escape the diacritics in any way. CustomSearch uses the use_multibyte System Setting to determine which set of string functions to use. It defaults to true, since most PHP installations include the mbString extension.

The 'properties' Field and Diacritics

As of version 1.3.0, you can search the properties field for strings with diacritic characters with no escaping in your search string. This was quite a challenge to code since those fields are in JSON format. JSON escaped the diacritics are escaped with a single backslash, but it requires four backslashes to search for them in a MySQL query. Because of the complexity, you can't do search and replace operations on the properties field with diacritics in the search term or the replace term. It's theoretically possible, but it's very unlikely that you would need this.

Security

In order to use CustomSearch in the Manager, users need to be logged into the Manager and have *all* of the following permissions: edit_document, edit_chunk, edit_plugin, edit_snippet, edit_template, edit_user, edit_tv.

These are all part of the Administrator Policy. If there are users you don't want to have access to CustomSearch, make sure they are missing at least one of these permissions in all Context Access Policies that are associated with their User Groups for the mgr context. Or better, create a custom permission for the CustomSearch menu item and give it only to users who should be able to use Custom Search.

Using CustomSearch in Code

CustomSearch includes the CustomSearchCodeExample snippets to show how you can use CS to perform searches and display the results in the front end. Because it's almost impossible for CS to sanitize user input and still find what you want, Do *not* allow users to enter search terms for use with a snippet unless you sanitize the input yourself before running the search — you have been warned!.

See the CustomSearchCodeExample snippet to see how it's done.

Because of the security issues involved, and the potential to trash the site, I would not use the search-and-replace option in the front end. To be safe, your code should abort if either of the doReplace or do_replace properties are set.

CustomSearch System Settings

Setting Description Default
cs_remember_settings Remember checkbox and max results settings Yes
cs_honor_searchable If set to 'Yes', Custom Search will skip resources that do not have the searchable field set False
cs_context_width Number of characters to show on each side of search term 25
cs_show_query If True, will show the Query used to search for each object False
cs_form_data Used internally for persistence of selections - DO NOT EDIT No default
cs_submit_on_enter If set, form is submitted when Enter key is pressed No

My book, MODX: The Official Guide - Digital Edition is now available here. The paper version of the book may still be available from Amazon.

If you have the book and would like to download the code, you can find it here.

If you have the book and would like to see the updates and corrections page, you can find it here.

MODX: The Official Guide is 772 pages long and goes far beyond this web site in explaining beginning and advanced MODX techniques. It includes detailed information on:

  • Installing MODX
  • How MODX Works
  • Working with MODX resources and Elements
  • Using Git with MODX
  • Using common MODX add-on components like SPForm, Login, getResources, and FormIt
  • MODX security Permissions
  • Customizing the MODX Manager
  • Using Form Customization
  • Creating Transport Packages
  • MODX and xPDO object methods
  • MODX System Events
  • Using PHP with MODX

Go here for more information about the book.

Thank you for visiting BobsGuides.com

  —  Bob Ray