Sitecheck CMP Tutorial

Overview

SiteCheck is both a diagnostic tool to help you find problems that are keeping your MODX sites from installing correctly, and a powerful set of methods that test the integrity of an entire MODX site.

SiteCheck is my first commercial extra. It took an unbelievable amount of time to create, and though I appreciate the donations people have made for my extras, I estimate that my hourly wage for creating them is well under a dollar per hour. I will still create and release free MODX extras, but I'm hoping that people won't mind contributing a little for a truly powerful tool.


Purchase SiteCheck

A SiteCheck License is for one person, but unlimited sites.

Features

Here's a partial list of the things SiteCheck will find:

  • Missing MODX config files
  • Invalid paths in config files
  • Incorrect file and directory permissions
  • Inability to connect to the MODX database
  • Character set mismatches between MODX and MySQL
  • An invalid workspace path
  • A Date/Time mismatch between PHP and MySQL
  • Invalid Transport Packages
  • Resources with invalid parents
  • Resources with invalid Templates
  • Resources with empty aliases
  • Resources with duplicate aliases in the same Context
  • Resources with empty URIs
  • Resources with duplicate URIs in the same Context
  • Resources with duplicate pagetitles in the same Context
  • Resources with context_key that does not match parent's
  • Resources with invalid class keys
  • Resources with invalid Contexts
  • Elements with invalid Categories
  • Elements with invalid Property Sets
  • Templates with invalid Template Variables
  • Template Variables with invalid templates
  • Template Variables with invalid Resource Groups
  • Resource Groups with invalid Template Variables
  • Resource Groups with invalid Resources
  • Resources with invalid Resource Groups
  • Users with invalid User Groups
  • Users with invalid Roles
  • User Groups with invalid Users
  • User Groups with invalid Roles
  • Plugins with invalid Events
  • Events attached to invalid Plugins
  • Property Sets attached to invalid Elements
On a typical site, SiteCheck performs over 5,000 tests.

Where SiteCheck really shines is the case where you move a site to another server or another directory and run Setup, but MODX doesn't run properly in the new location. Often, it's a problem with the file paths, the workspace path, or the file permissions in the new location. SiteCheck will tell you exactly what's wrong.

Even if SiteCheck doesn't find any issues with a particular site, it will give you a warm feeling to know that the site has no hidden problems. You'll also have a valuable tool to use when things go wrong in the future or when you want to make a quick backup of your MODX database.

SiteCheck includes a convenient Database Backup utility that can save multiple, rolling backups of your entire MODX database.

History

SiteCheck began as a simple extra designed to help people who were having trouble installing MODX. It provided helpful diagnostic messages about invalid paths, missing critical files, and incorrect file and directory permissions. Once that all worked, I added the capability to test the database connection and to detect character set problems.

At that point, I realized that SiteCheck could also help people whose MODX installs were not working correctly for a variety of reasons. This capability expanded to include integrity checks for all basic MODX objects on the site: Resources, Snippets, Plugins, Chunks, Templates, Template Variables, Property Sets, Intersects, Transport Packages, Resource Groups, Users, and User Groups.

Site Corruption

Over time, most MODX sites become corrupted in some way. Typically, this is not the fault of MODX or any user, but is caused by extras that don't install or uninstall correctly. For example, Resources can end up with invalid parents and elements can be placed in invalid categories. Sometimes, transport packages no longer have their source transport.zip files. These invalid objects are still in your database. The Resources and elements usually won't show up in the Manager at all, so you can't correct or delete them, and if you try to create something with the same name, you'll get a mysterious error message. They can also cause weird problems that are hard to diagnose and even harder to solve.

Another side effect of invalid objects is spurious error messages showing up in the MODX Error Log — sometimes, quite a lot of them. Important error messages may be lost in the clutter, and your Error Log may grow so big that MODX can't display it.

Even when the invalid objects don't cause problems you can see, they are still slowing down the site by cluttering up the database and sucking up memory.

SiteCheck not only finds these problems, whenever possible it also fixes them. With Transport Packages, for example, it will correct invalid fields and (if necessary) download the transport file from the MODX repository.

When SiteCheck can't fix a problem, it will provide information (either directly or with a link) to help you solve it. SiteCheck has a "Fix" option. When "Fix" is off, SiteCheck will report the problems without making any changes. When you select the "Fix" option, it will attempt to fix any problems. For safety, whenever SiteCheck intends to modify your database, it will do a full backup of the database first.

Installing SiteCheck

SiteCheck has to run in a variety of ways. Because it helps diagnose problems in installing MODX, it has to run without MODX installed. As a result, it has several installation methods — none of them typical. It's very easy to install, but it can't be downloaded directly into Package Manager (though once MODX is installed, you can install SiteCheck in Package Manager).

When you purchase SiteCheck, you'll get a downloaded .zip file named for the current version. You'll also get an email with a link to download the file again in case there was a problem with the original download.

The sitecheck .zip file contains both the raw files used by SiteCheck, and the transport package to install it in Package Manager. Simply transfer the sitecheck .zip file to your MODX root directory (the one with the assets, core, and connectors directories). Unzip it there. You can generally do this in cPanel. Be careful. Some unzip programs will want to put the file in a sitecheck . . . directory. Be sure to set the target to the MODX root directory by deleting sitecheck . . . from the target path.

After the unzipping, you should see a core/components/sitecheck directory and an assets/components/sitecheck directory. You should also see the SiteCheck transport.zip file in your core/packages/ directory.

At this point, SiteCheck will run, but outside of MODX. If MODX is installed and working, go to System | Package Management. Click on the down arrow next to "Download Packages" and select "Search Locally for Packages" (click on "Yes" in the popup dialog). You should see SiteCheck in the Package Manager grid and you can install it as usual by clicking on the "Install" button.


If you want to use Sitecheck on an already working site, unzip the package somewhere locally (it doesn't need to be in a MODX install) and transfer the .transport.zip file from the resulting local core/packages/ directory to core/packages/ directory at your site. Search Locally for Packages on your site as described above, and install it with the "Install" button.

Upgrading SiteCheck

When a new version of SiteCheck is available, you'll get an email announcement. The new version should be attached to the message and there will be a download link for it as well.

Upgrading works just like installing. Transfer the .zip file to your MODX root directory and unzip it there. The new version should appear in Package Manager when you "Search Locally for Packages" and you can install it by clicking on the "Install" button.

If you have a working MODX install, you can also extract the files from the .zip file anywhere and copy the extracted core/packages/sitecheck...transport.zip file to the core/packages directory of any MODX Revolution install, then in Package Manager, search locally for packages and install SiteCheck.

Running SiteCheck in MODX

If you have installed SiteCheck through Package Manager, you can run SiteCheck from the "Components Menu". If you have just installed it, you may have to reload the Manager page before the menu option will show up. Select the test or tests you want to run and click on the "Perform Tests" button. SiteCheck will display its output below the form. You may have to scroll down to see it. There are a lot of tests, so be patient.

On some hosts, SiteCheck may time out. If so, you may have to run the tests one at a time. In fact, that's not a bad technique in general. It helps to make sure that you get the problems straightened out without forgetting any of them. If SiteCheck finds problems, you may want to paste the results into a text editor so that you'll have them available while you're working on the problems.

The default PHP timeout is 30 seconds, which may not be long enough for SiteCheck to run, depending on the speed of your server and how big your site is. Some hosts will increase the timeout for you on request. If things are slow enough that some tests won't finish before the timeout, even when you run them one test at a time, and the host won't help, you have the option to run SiteCheck from the command line, where the timeout won't apply.

If you select the "Verbose" option, you'll see more detail in the output. If you select the "Fix" option, SiteCheck will attempt to fix any problems. It's recommended that you always run SiteCheck *without* the "Fix" option the first time through to get a look at what the problems are.

SiteCheck Colors

If SiteCheck is running in a browser, progress reports and general information are shown in black, errors found are shown in red, success messages (no problems) are shown in green, and actions taken by SiteCheck are shown in blue.

For all Resources and Elements (except property sets), when you are logged in to the Manager, the names of each problem object will be displayed as a link to the Manager page for editing the object. The link will open in a new window or tab so you won't lose the SiteCheck output. This allows you to fix problems in the Manager for Resources and Elements, which is often easier than using the "Fix" option for them.

Running SiteCheck Outside of MODX

If MODX won't install, you'll want to run SiteCheck without it to help you find out what the problems are. You can run SiteCheck in a browser by entering the following URL in the browser:

https://yoursite.com/core/components/sitecheck/index.php

The URL above won't work if you have hardened your core directory with an .htaccess file — you'll get a message telling you that you're not allowed to access that directory. In that case, you can use this URL as an alternative:

    https://yoursite.com/assets/components/sitecheck/index.php

In both cases, you can usually leave off the index.php at the end of the URL, since that will run by default.

Which tests are performed will depend on the settings at the top of the core/components/sitecheck/elements/snippets/sitecheck.snippet.php file. The defaults are probably fine, but you can edit them if you want to. Each option is set to true or false (no quotes). The "Verbose" option is off as it the "Fix" option.

On a new site, SiteCheck will run very quickly because there are no Elements and only one Resource, so you shouldn't have to worry about the timeout setting.

The "Fix" option won't do anything until MODX is installed because all the fixes require MODX. Problems detected when running SiteCheck without MODX installed will have to be fixed manually, but knowing what the problems are can be a tremendous advantage.

Once you've fixed any problems, you should be able to run Setup and install MODX. Once Setup has finished, you can run SiteCheck directly from your browser again before trying to login to make sure everything is all right. You may see a PHP warning about not being able to load the namespace metadata. You can ignore that.

If you see an error regarding the date/timezone setting, that can be fixed in the System Settings grid in the Manager once you log in. Click on the link in the error message for more information.

If everything checks out, you should be able to log into the Manager, and then install SiteCheck through Package Manager as described above.

At this point, it's not a bad idea to set all the options to false (no quotes) in the sitecheck snippet file if you won't be running SiteCheck outside of MODX.

core/components/sitecheck/elements/snippets/sitecheck.snippet.php

That will keep SiteCheck from displaying any information at all to anyone who might figure out how to run it from the outside and it won't affect how SiteCheck runs in the Manager.

It you intend to continue using SiteCheck outside of MODX (it runs somewhat faster, and from the command line it will not time out), leaving the variables set to true is not all that bad as long as you leave the verbose and fix options set to false. No sensitive paths will be displayed in the output and nothing on your site will be modified. SiteCheck will load MODX if possible, so the "Fix" option will still work.

In some situations, the SiteCheck CSS file may be lost if SiteCheck finds certain errors. The output won't be as pretty, but the information will still be there.

You also have the option to run SiteCheck from the command line by executing the index.php file in the core/components/sitecheck/ folder:

php path/to/modx/core/components/sitecheck/index.php

Database Backup

If the "Fix" option is selected, SiteCheck will attempt to back up the database by creating an .SQL file in the core/components/sitecheck/backup directory. The backup will be made even if no tests are selected, as long as the "Fix" option is checked. That makes for a handy way to create a full backup of your MODX database at any time. Just run SiteCheck with only the "Fix" option checked.

The maxBackups System Setting determines the number of database backups SiteCheck will keep. The default is 5, but you can set it to whatever you like. Each time you run SiteCheck with the "Fix" option, a new backup is created. (In fact, you can use SiteCheck to create a full database backup by selecting any test and the "Fix" option). When the number of backups reaches the limit, the oldest backup will be removed. The backup files are prefixed with the time and date of the backup in the format: YYYY-MM-DD--HH-MM-SS.

The mysqldump utility is used to create the backups, and it may require some configuration. If your host has the mysqldump executable in a non-standard location, and/or no alias is set up for it, SiteCheck won't find it. In that case, you can set the first term of mysqldumpCommand System Setting to contain the correct location.

Before changing the System Setting, check the permissions of the core/components/sitecheck/backup/ directory to make sure it's writable.

The default value of the mysqldumpCommand System Setting is:

mysqldump -u{db_user} -p{db_password} {database_name}

Generally, the only thing you need to change if it doesn't work is the mysqldump part. You can check with your host or the host's forum for the correct path to the mysqldump utility. The user, password, and database name are replaced automatically by SiteCheck so you should never change those (assuming that they are correct in the config.inc.php file, which they are if MODX runs).

Here are some possible alternatives for the first term of the command:

/usr/local/mysql/bin/mysqldump
/usr/bin/mysqldump
/usr/local/bin/mysqldump
/usr/mysql/bin/mysqldump

On some servers, you may get an error message saying that you don't have permission to access the mysqldump utility or that it can't be found, even though the backup is successful. If you get the error message, check for the file and make sure it's not empty. If it's there but empty, that means that the path to mysqldump is correct, but there's something wrong with format of the rest of the command.

I have seen suggestions on the web for this format, but I've never encountered a server that used it:

mysqldump --u={db_user} --p={db_password} --host=localhost {database_name}

If you can't get the database backup to work, you can still have SiteCheck fix errors by setting the proceedWithoutBackup System Setting to Yes. SiteCheck's fix operations are fairly benign and conservative, so don't be afraid to use this option. You can always back up the database yourself in PhpMyAdmin first if you want play it safe. Remember that for Resources and Elements, Sitecheck provides links to edit them whenever the problem can be fixed that way. Clicking on a link in the error message and correcting the problem directly is often easier than using the "Fix" option.

If it takes you a number of tries to get the mysqldump command correct, you may have a number of extra backup files (beyond the "maxBackups" number) in the backup directory that need to be deleted manually. The number of files won't grow if it's beyond the "maxBackups" number, but only the oldest file will be deleted.

If you need to restore your database, download one of your backup files to your local machine, either on the MODX files tab or in cPanel. Take the site offline temporarily. Then, go into PhpMyAdmin in cPanel and select the MODX database. Select the "Import" tab at the top of the screen, browse to the backup .SQL file on your local machine and select it. Click on the "Go" button to restore the database.

For a quick database backup, just run SiteCheck and select "Check Workspace" and "Attempt to fix errors." This is the fastest test and will give you a full database backup in short order.

A Note on Obfuscation

Because SiteCheck has to be able to run from a browser outside of MODX and there's no way for it to tell if it's being run by you or someone else, it's a little cryptic about the paths. It describes them generically, or in terms of the MODX constants, like MODX_CORE_PATH and MODX_CONFIG_KEY. It there is a problem, SiteCheck will display the actual path. If not, you'll see messages like these:

Checking Paths in main config file
Checking MODX_CONFIG_KEY.inc.php
Checking settings in MODX_BASE_PATH config.core.php
Checking settings in MODX_CONNECTORS_PATH config.core.php
Checking settings in MODX_MANAGER_PATH config.core.php

This is done to prevent outsiders from seeing the names of these critical files and directories. In a default install of MODX, any reference to the "main config file" or "MODX_CONFIG_KEY.inc.php" refers to the core/config/config.inc.php file (the default definition of MODX_CONFIG_KEY is "config"). The MODX_BASE_PATH refers to your root directory, MODX_CONNECTORS_PATH refers to the connectors/ directory in the root, and MODX_MANAGER_PATH refers to the manager/ directory in the root.

Resource Tests

SiteCheck will fix some of the problems with Resources automatically. It will correct an invalid class key or an empty alias or URI. In the case of Resources with an invalid parent, it will put the resource in the "SiteCheck Problem Resources" folder and tell you that it's done so. You can drag them from there to where they belong in the tree.

Rather than use the "Fix" option, you can fix some Resource problems (e.g., an invalid parent or template) by clicking on the link in SiteCheck's error message and fixing the problem yourself if you are running SiteCheck in MODX.

You may remove the "SiteCheck Problem Resources" folder if you like when all problems are fixed, but make sure there are no resources in it or they will be deleted too. If SiteCheck needs the folder during a future test, it will be re-created. Be sure to empty the trash after deleting the folder (using the trashcan icon at the top of the Resources tree), otherwise, SiteCheck won't be able to re-create it and some Fix options won't work.

When resources have duplicate aliases (if you have the allow_duplicate_aliases System Setting set to "No") or URIs, SiteCheck can't fix the problem automatically because it has no way of knowing what you want. It will simply tell you about them and you'll have to fix them manually. Resources with duplicate pagetitles are not necessarily a problem, but it's possible to have two resources with the same pagetitle when you only want one, so it reports them.

Element Tests

In the case of Elements, the only problem (other than ones that show up in the Intersect Tests) is Elements with an invalid Category. This happens when the Element is in a Category that no longer exist. These Elements don't show up in the tree, so without SiteCheck, you would never know they existed unless you spent a lot of time looking at the tables for all elements in PhpMyAdmin (where the Category is a number, not a name). SiteCheck will put these elements in the "SiteCheckProblemElements" Category and tell you that it's done so. You can then find them in the Elements tree and set their categories correctly by editing and saving each one.

Note that the elements will still be under their element type in the tree. So problem Chunks, for example, will be under Chunks | SiteCheckProblemElements and problem snippets will be under Snippets | SiteCheckProblemElements. In older versions of MODX, elements were also listed in the Category part of the tree, but this is no longer the case.

You may remove the "SiteCheckProblemElements" category if you like, once any problems have been fixed. If SiteCheck needs it during a future test, it will be re-created. If there are any elements in that category when the category is deleted, they will not be removed. Instead they will have no category, but will still show up in the tree.

As with Resources, you can also click on the link in SiteCheck's error message when running in MODX to edit the element directly.

Intersect Tests

Most of the tests performed by SiteCheck are self-explanatory, but the Intersect tests need a little explanation.

MODX objects often have many-to-many relationships. A Resource, for example, can be in more than one Resource Group and a Resource Group can contain more than one Resource. In these cases, MODX keeps the information about the relationship in a separate table which stores the intersect object. In the case of Resources and Resource Groups, it's the modResourceGroupResource object, which stores the ID of the Resource and the ID of the Resource Group it's connected to.

Intersects can get corrupted when an object is deleted but its corresponding intersect object is not. This is more common in earlier versions of MODX, but you may have some invalid intersects left over, because once they exist they never get deleted. They can make things fail to show up in the Manager grids, slow down the site, and cause mysterious problems when MODX won't let you create a new relationship because an invalid one already exists.

SiteCheck will show you all the invalid intersects, and with the "Fix" option selected, will remove almost all of them (more on this in a bit). Before fixing them, though, it's a good idea to take a look at the SiteCheck output because there may be clues about how the objects are meant to be related.

For example, suppose there is an invalid modResourceGroupResource because the Resource Group no longer exists (or has a different ID). The intersect is useless and should be removed, but the error message suggests that that Resource belongs in some Resource Group (SiteCheck would fix this, but it has no way of knowing which one). You might want to see if that Resource is not in any Resource Groups it should belong to.

Let's look at the reverse case: a Resource Group with an invalid Resource. SiteCheck can't tell you the name of the Resource because it doesn't exist, but you might want to look over the Resources in that Resource group and see if you notice any Resources that should be there but aren't.

There are a couple of cases where SiteCheck won't remove a bad intersect (and it will report that the intersect was not removed). If a modPluginEvent is valid except for an invalid Property Set, SiteCheck won't remove it because it successfully connects a Plugin to a Plugin Event. The report will tell you the name of the Plugin and the Event so you can easily fix the problem. Just edit the Plugin, and on the System Events tab, find the named Event. Double-click on the Property Set field and select the correct Property Set. If you don't see it in the drop-down, go to the Properties tab, click on the "Add Property Set" button, and select the Property Set. Then, go back to the System Events tab and select the property set. Important: Be sure to save the Plugin when you're done.

Similarly, SiteCheck won't remove a modUserGroupMember intersect if the User and the User Group are valid but the Role is not. The report will give you the name of the User Group and the username of the User. Go to Security | Manage Users, right-click on the User and select "Update User". On the "Access Permissions" tab, Right-click on the line for the User Group and select "Update User Role". Select the proper role in the dialog and click on the "Save". Important: Click on the "Save" button at the upper right to save the user.

Transport Package Tests

SiteCheck checks the integrity of each transport package for both internal consistency and the existence of an external transport.zip file. It reports any errors it finds, and will attempt to download missing transport files from the MODX Repository.

For example, a package may have an invalid signature. This is sometimes the result of "Search Locally for Packages" finding non-transport files in the core/packages/ directory and misidentifying them as transport packages. Packages with an invalid signature are generally useless and SiteCheck will delete them if the "Fix" option is on and it can't correct the signature, but only if the package is not installed.

If the deleteInvalidPackages System Setting is set (it's off by default), SiteCheck will delete any packages it can't download from the repository.

It is a quirk of Package Management that packages installed using "Search Locally for Packages" or as subpackages (like Quip and tagLister that are installed as part of Articles) end up with all lowercase names in the Package Manager grid, regardless of their "real" names. It may be annoying, but as far as I can tell, has no effect on how the packages function. I did not find any way to correct them with Sitecheck.

Note also that packages installed with "Search Locally for Packages" will not check for updates or display the "Check For Updates" button. This can be corrected (if they are available from the MODX repository) by changing the provider field in the modx_transport_packages table to the ID of the default provider (usually 1). SiteCheck won't do this for you because if the package is not available from the MODX repository, it will generate a lot of errors in the Error log. For SiteCheck itself, the correct provider ID is 0 because it's never downloaded from the repository.

SiteCheck System Settings

Setting Description Default
createAliases Create aliases automatically for Resources with empty aliases Yes
deleteInvalidPackages Remove packages from DB if file can not be downloaded (packages with an invalid signature will be deleted regardless of this setting unless they are installed). No
maxBackups Determines the number of database backups that SiteCheck will keep. If the number goes over the limit, the oldest backup will be removed. 5
proceedWithoutBackup Tells SiteCheck to fix errors even if the database is not backed up. No


You can edit the System Settings above by going to System -> System Settings on the Manager's Top Menu. Change the drop-down that says "core" to "sitecheck". All System Settings will survive an upgrades or reinstall of SiteCheck

The createAliases System Setting determines whether SiteCheck automatically creates aliases for Resources with a blank alias. By default, it's set to "Yes". If you would prefer to create the aliases yourself, you can turn it off by setting it wo "No."

The deleteInvalidPackages System Setting determines whether packages that can't be downloaded from the repository are deleted from the database. The default setting is "No" because they might be packages that you can get somewhere else (like SiteCheck itself). Packages with an invalid signature will be deleted regardless of this setting, since they are useless. If you know a package should be deleted from the database, change the System Setting to "Yes," and run SiteCheck again with the "Fix" option. The package will be deleted. It's recommended that you set the System Setting back to "No" after doing so.

The SiteCheck snippet has only one property, sitecheck_default_tests. It determines which tests are checked when you run SiteCheck in the Manager. It's set automatically to whatever tests you checked the last time you ran SiteCheck. You should never need to edit this property yourself.

 

My book, MODX: The Official Guide - Digital Edition is now available here. The paper version of the book is available from Amazon.

If you have the book and would like to download the code, you can find it here.

If you have the book and would like to see the updates and corrections page, you can find it here.

MODX: The Official Guide is 772 pages long and goes far beyond this web site in explaining beginning and advanced MODX techniques. It includes detailed information on:

  • Installing MODX
  • How MODX Works
  • Working with MODX resources and Elements
  • Using Git with MODX
  • Using common MODX add-on components like SPForm, Login, getResources, and FormIt
  • MODX security Permissions
  • Customizing the MODX Manager
  • Using Form Customization
  • Creating Transport Packages
  • MODX and xPDO object methods
  • MODX System Events
  • Using PHP with MODX

Go here for more information about the book.

Thank you for visiting BobsGuides.com

  —  Bob Ray