Best Practice Guide on How to Migrate Data Between cPanel Servers

Dealzclick

March 1, 2022

2 Views

SaveSavedRemoved 0

[ad_1]

In 2018, I had the opportunity to speak at the cPanel Conference in Houston.

I presented an in-depth look at strategy and techniques for migrating data between cPanel servers at all scopes.

This included the ways to prepare for a server migration to ensure site functionality, security, and performance upon arrival, as well as account completeness, common sense defaults, and pitfalls.

Since then, migrations have not changed much, but they have changed. I revisited and revamped this article to ensure that it still meets the needs of migrators today.

It is also important to note that CentOS 6, on which many cPanel servers are run, is going to go End Of Life at the end of February 2020.

This means it will no longer receive any security patches, and a migration to a new server running CentOS 7 or higher will be necessary in order to maintain PCI compliance and general server security. These instructions should be very suitable for guiding you through that upgrade.

Enjoy!

Considerations for cPanel Migrations

Often, the focus during cPanel migrations is on the wrong elements. The most common request is to ensure that every file and database is copied over to the new server accurately, which by itself is a relatively trivial task. However, the real magic of migrations is in ensuring service compatibility, which will allow sites to load on the new server just as they did on the old server.

One of the first things to do when considering a migration is to think about your DNS. For most migrations, particularly those which technology or host platform changes, a change in IP address is needed. Because of this, you will need to understand where your nameservers lie for each domain involved. You should also take the time to lower the TTL values for your zone files as low as possible, so that when the time does come to update DNS, changes will propagate quickly.

Another item to consider is where your nameservers are hosted. If they are on the source machine (the one you are migrating from), then once it is terminated, the nameservers will no longer be accessible, so plans should be made to move the DNS to the new server.

The sites to be migrated can also be prepared for migration in an optional step. Take out the trash to reduce the quantity of data to migrate, and update any sites that are using an out-of-date Content Management System (CMS). This will help ensure compatibility with new default minimum versions on new servers, just in case software versions on your old server are no longer available for install.

Like this content? Subscribe now to get more tips for optimizing infrastructure sent straight to your inbox.

Ensuring Compatibility

Now, work can begin on ensuring compatibility, a critical step in migration. A good look should be taken at the core services of the servers and determine whether the upgrade path will remove or change any functionality. Some of this determination can only come with experience and knowledge of these core services, but generally your CMS vendor will be able to tell you their recommended minimum or maximum versions of certain software.

For instance, modern WordPress has a recommended PHP version of 7.2, but will work with versions as old as 5.2.4. However, very old versions of WordPress are programmed with functions that are not compatible with PHP 5.6 or greater, which is why updating your code and plugins is recommended when starting a migration. This is because PHP 5.6 and below are End Of Life (EOL) software, and no longer receive security updates, so upgrading to something modern and supported is a good idea.

With Managed WordPress from Liquid Web, all platform and plugin updates happen automatically, so you don’t have to worry about them.

Some websites have strict requirements for older versions of software, and upgrades are not always possible. For these types of legacy applications, support for End Of Life software is necessary, so consider adding CloudLinux to your server, which, among other features, allows installation of hardened PHP versions as old as 4.4 to augment those available through your control panel.

What Specifically Should Be Checked First, and How is All of this Checked on Your Source Server?

Many favor to use the command line to investigate this information:

cat /etc/redhat-release
df -h
df -i
du -hs /var/lib/mysql
cat /proc/cpuinfo
free -m

These commands will check, in order, the version of CentOS on the server, indicating the general age of the system; the total disk usage of all partitions; the inode (file and folder) count; the disk usage of the /var/lib/mysql folder, as well as its subfolders, hinting at any very large databases; the processor architecture and core count; and the total memory on the server, in megabytes.

This look at hardware helps us size up the target system and make sure there is enough room and power for the data we want to move. It will also make sure that we are not severely altering the server’s capabilities unexpectedly, such as halving the available memory on database-intensive applications.

LAMP Software

This set of commands will look at the remainder of the LAMP stack:

php -v
php -m
httpd -v
httpd -L
httpd -S
mysql --version

Here, the output shows the primary PHP version and its installed modules; the Apache version, modules, and running sites; and the MySQL version. These are compared to similar output from the target server, and matched up where possible. As we discussed before, PHP version availability will be important to ensuring functionality, but the Apache and MySQL versions are not nearly as critical.

Because WHM utilizes EasyApache to make things easy, the act of copying a running EA configuration into another server is made even easier by using a certain set of scripts. When utilizing EasyApache 4 on the source server, the following command will export a single JSON file listing out all of the installed packages:

/usr/local/bin/ea_current_to_profile --output=/home/ea4-migration-profile.json

If you happen to be using the older EasyApache 3, the necessary information is stored in a YAML file instead, and can be converted using a utility on the new server, which should be running EasyApache 4. Bring the file /var/cpanel/easy/apache/profile/_main.yaml over to the target machine, and then use this command to turn it into an EasyApache 4 profile JSON file:

/scripts/convert_ea3_profile_to_ea4 /home/_main.yaml /home/ea4-migration-profile.json

Either of these output files can then be restored as a profile on the target server (if it is using EA4) by using this command:

ea_install_profile --install /home/ea4-migration-profile.json

For good reason, WHM disallows downgrades of the MySQL server, but generally, database dumps from mostly any version can be made to work with modern MySQL. The issue that gets encountered more frequently is old password hash support being removed in version MySQL 5.6 or greater; passwords stored in the old hash format will have to be re-entered manually before they can work.

WHM Settings

Just as your clients would expect consistent behavior from the server, you would expect consistent behavior from WHM. So, the settings in WHM will need to be researched and duplicated. WHM has a built in tool to help with this import and export procedure that can be leveraged to copy the configuration for one WHM server into another.

/usr/local/cpanel/bin/cpconftool --backup

The backup is stored in the /home/ directory on the old server, and can then be copied to the new server and restored with this command:

/usr/local/cpanel/bin/cpconftool --restore=/path/to/filename.tar.gz

There are other items outside of this backup script that should also be attended to, including the Packages and Features levels created inside of WHM, Communication preferences, Backup preferences, and any Plugins installed in WHM, like ConfigServer scripts.

Packages and Features are rather easy to move: their respective folders can be directly copied to the new server in the appropriate location. These two commands run on the target server will accomplish that, assuming 123.45.67.89 is your old server’s IP address:

rsync -aqHz 123.45.67.89:/var/cpanel/packages /var/cpanel/
rsync -aqHz 123.45.67.89:/var/cpanel/features /var/cpanel/

Communication and Backup preferences should be checked from inside the WHM interface. Though they are both available from the command line, the commands are a bit outside the scope of this post, and they are simple enough to copy manually. Any plugins to WHM should be reinstalled fresh on the new server.

Other Services

With Linux, it is possible to install anything you want anywhere on the system you want, making researching extra software or services rather difficult. Ideally, you will already know what software your sites depend on, and what is installed on the source server to support them, though this is not always the case.

There are a few important commands to remember when trying to find ‘other stuff’:

The first command shows all running processes on a server, which can be selectively parsed and grepped for particular lines such as Varnish or cPHulk. The second command shows listening services, which should export a few dozen lines that can be scanned for alternative listeners like Nginx, Litespeed, or Memcache. This command also shows services that are using nonstandard ports, like Apache on 8080 as a backend to another server, or a custom Java application listening on its own port.

Knowing exactly what to look for on the output of these commands is a matter of experience, but looking for running programs that are not running on the target server (and installing them where needed), as well as services that are listening to nonstandard ports (and configuring them on the new machine), is a good place to start.

LAMP Configurations

There are a variety of settings that can be modified from within WHM in regard to MySQL, Apache, and PHP. It is not recommended to copy the configuration files for which the settings are saved outright to the new server, as there may be minor variations between builds or versions that can cause these applications to not start.

There is also the matter of changes in resource levels. If an upgrade in servers is being performed, then larger or smaller limits may be necessary to avoid memory issues or to utilize hardware effectively.

To be safe, I recommend that any minimum variables be set for your site software, and then tuning be performed after the new server is taken live.

Duplicating Accounts

Now that the environments are in sync, we have finally arrived at the core commands for moving data!

There are two core commands that need to be run to restore the accounts, and in essence we are packaging up the needed cPanel accounts, copying the backups, and restoring them on the new server. But, to save space and time, we will use a few extra tricks.

First, it is recommended to making a working directory:

mkdir /home/migrationtemp

For each user (we will use username in our example), the following three steps will need to be taken:

/scripts/pkgacct --skiphomedir username /home/migrationtemp/

This will package up the account, skipping the very large home directory, and place the backup in /home/migrationtemp/.

Why Skip the Home Directory?

It takes a lot of CPU resources and disk space to copy the home directory into the backup, compress it, then copy it over to the new server, decompress it, and copy it once again into the appropriate location. Since the folder is not changing the structure, the home directory can be copied directly to the new server separately and directly, skipping all the compression steps, and saving disk space on both servers.

Once the backup is ready (it should be very quick), it can be copied to the new server using your favorite method (for single files, I prefer scp), and then restored:

/scripts/restorepkg /home/migrationtemp/cpmove-username.tar.gz

This will bring back the entirety of the cPanel account on the new server, save the home directory. Let’s address that now. This command is run on the new server:

rsync -avHPz 123.45.67.89:/home/username /home/

Make sure that you pay attention to those trailing slashes! This command syncs the /home/username folder and its contents into the /home/ folder on the new server, preserving permissions, acting verbosely, and compressing data.

We also have more information on the rsync command.

We’re done! Now it’s time to test your migration.

Testing Your Migration

There is really only one good way to test a migrated website accurately: by altering the hosts file on your workstation.

This will essentially ‘trick’ your browser into thinking that DNS is already updated by bypassing nameserver lookups. Public traffic will continue to be routed to the original server, but your own computer will be browsing the new server. And, because the new server is configured to accept requests for the real domain name as soon as the cPanel account is restored, it will respond as is would as if it really were live. As far as your computer and the new server are concerned, this is ‘real’ traffic.

We have an article available for modifying your hosts file that is geared towards several different operating systems, since the procedure varies by version and vendor.

What Are You Testing For?

Anything weird.

I recommend looking at the main page, a few different sub pages, pages that parse important information or have high visibility, order forms or carts, and the backend of your site as well.

You can be as thorough here as you like, and if you should run into any issues, you can follow normal troubleshooting techniques for the target server. Knowing that the errors are not yet live or public really takes the stress off! Take as much time as you need and make sure the new server is working exactly how you want before we proceed to the next stage.

The Final Sync

For cPanel to cPanel migrations with root-level SSH access, which is typical for server owners, updating any changed information is very straightforward. New database dumps should be taken and imported, and the home directories should be resynced one more time. This is all followed by a DNS update.

As you recall, we looked at the inode count on the old server as well as the disk usage of the MySQL directory /var/lib/mysql. These two items together help us project the length of time the final sync might take. Unless you have databases that are over, say, 50GB, or /home directory inodes well over 2 million (indicating a lot of files to compare), a final sync can normally be executed in less than an hour, or two at most.

Because the point of the final sync is to bring everything up-to-date on the new server that may have changed during testing, it is important to ensure that no further changes are made on the origin server, either by content creators (posts) or by visitors (orders or comments). This will aid in the accuracy of the content that is made live on the new machine. Therefore, it is best to interrupt services that allow sites to be visited, incurring downtime to the websites and email.

These commands will stop the necessary services on the old server:

/usr/local/cpanel/bin/tailwatchd --disable=Cpanel::TailWatch::ChkServd
service httpd stop
service cpanel stop
service exim stop

This will prevent websites from being viewed, email from being delivered, and cPanel or WHM from being accessed. You can see why it is important to select a time when traffic will be low! MySQL will need to be kept running, as we will need it to perform database dumps.

There are now just a handful of commands to consider, all of which are run on the target machine:

rsync -avHPz 123.45.67.89:/home/username /home/ --update
mysqldump username_db | gzip > \ /home/migrationtemp/username_db.sql.gz
ssh -C 123.45.67.89 “mysqldump username_db” | mysql username_db

The first command reuses our original rsync command, but adds the –update flag, which will compare timestamps on files to be synced and only copy those files which are newer on the source server.

The next command backs up the database on the target server to a zipped file, just in case something goes awry, and the final command will stream a dump of a database called “username_db” from the old server directly into the new server.

Streaming the database dump over SSH is much more efficient on space and task time versus taking database dumps and copying them over, then importing them. This single line takes all three of those steps and combines them into one, which is critical when you have forced your sites offline for the sync.

Update DNS

The final step in the migration is to let your visitors know to access the new server. This is done with a change in the A record in your DNS zone files, wherever they may be. Since we lowered the TTL values at the start of the migration, this new A record should propagate around the internet very quickly.

Once DNS is updated and propagated, your new server will be serving content, and your migration is complete! Make sure to turn on your local and/or remote backup solution, and terminate your old server. If you need to move your nameservers off of the old server, updates at the domain registrar will be needed to change the glue records, or look at changing to an off-server solution by using Liquid Web’s nameservers.