File Synchronization Between Several Application Servers in a Cluster
As incoming traffic to my project grows, I need to improve my app performance to keep up with it. Some room of capacities for elastic extension is automatically provisioned in confines of a set server vertical scaling limit, but eventually I most likely will face the need to enlarge a number of its nodes (i.e. to scale it horizontally) upon my service popularity rises.
For lots of the most widely used apps (like WordPress, Drupal, Joomla, Liferay, Redmine and similar ones) such distribution across multiple web server instances becomes a real problem, as usually all user-provisioned content is saved only on a node that has processed the corresponding file upload request.
To help me in getting rid of such content being out-of-sync upon building clustered solution for my application, the CloudJiffy PaaS provides a special File Synchronization add-on, intended to keep the uploaded files set similar across multiple web-server instances. Herewith, synchronization could be applied not just in confines of one environment (i.e. for a single horizontally scaled node) but even across application servers in two separate environments, regardless of whether they run the same stack and/or application or different ones.
- Install Example WordPress Application
- Test File Upload Without Synchronization
- Apply File Synchronization
- Check Sync Logs
File Synchronization Add-on Implementation
The File Sync solution is provided as a single-click add-on, implemented with a bundle of lsyncd daemon and cron utility. Herewith, being wisely coupled with inotify, lsyncd initiates file synchronization only when any actual changes are detected on the system. Such realization notably offloads CPU in comparison to the regular sync calls and keeps my data up-to-date all the time, which makes this solution simultaneously simple to use, powerful, and affordable.
The File Sync add-on can be installed on top of any CloudJiffy-managed application server stack (i.e. except for Docker containers). Before the appliance, I will be asked to select the folder to synchronize. In this way, I am able to sync only the part of my app server’s file system that actually requires this. Thus, only changes within the selected directory (on any of specified application server instances) will trigger the synchronization.
Now, let’s see how to put all of this into action step-by-step.
Install Application
In our example, we will use the WordPress application, deployed to an environment with two NGINX-PHP application servers.
Click Install and provide the required details, such as Environment, Display Name (alias) and Region (if several regions are available).
Confirm the installation and wait a minute for a message with my admin data to be shown (the same information will be simultaneously sent to me via email).
When finished, click Apply.
Tip: By using instructions in our documentation, I can easily add extra app servers, enable high availability or even configure a clustered solution.
Test File Upload Without Synchronization
At this point, let’s check how the file upload is processed without synchronization.
2. Once logged in, make some custom changes (e.g. upload images or other files, edit themes, add pages, etc).
As an example, we’ve uploaded one image to the Media library.
4. Navigate to the webroot (click from Favorites) > ROOT > wp-content directory and compare its content within both nodes.
Note: The directory with uploaded files depends on the application I am running. Here is the list of such folders for some widely-used applications:
- WordPress - webroot/ROOT/wp-content
- Drupal - webroot/ROOT/sites
- Joomla - webroot/ROOT/images
- Liferay - webroot/home/liferay/data
As I can see, one of the app server instances (Node ID 54502 in our example) contains the newly added file in the uploadsdirectory, whilst another one (with 54505 identifier) does not have it yet.
Apply File Synchronization
So, to implement content synchronization inside our cluster, let’s apply the File Synchronization add-on.
Here, specify whether file synchronization should run within One environment (as in our example) or between Two different environments (see the related dedicated options in the table below),. Apart of that, provide the following details:
- Sync path - path to a folder (relatively to the webroot directory) that should be synchronized on a chosen below (within the Nodes field) server, e.g. /ROOT/wp-content
- Sync delay (sec) - synchronization delay, stated in seconds
- Environment name - to select the required environment from the list of available ones
-
Nodes - the app server layer is fetched automatically upon selecting an environment
When I are finished, click Install.
- 2nd environment IP - address of any application server node within the second environment (I can use internal IP for the case both environments are run on the same Platform and hardware region; otherwise, external IP address is required)
- Password - optional box to set a password on synchronization (should be stated the same on both instances)
4. At this point, I can check the uploaded file’s presence within both nodes.
As I can see, the added image is available at the each node’s uploads directory now.
It’s that easy! File synchronization will be automatically performed every time I add or change anything in my WordPress (or any other application I maintain) so that data on my servers will always be identical.
Check Logs
The most important information and status of synchronization can be viewed in the lsyncd.log and lsyncd.status files.
Hopefully, this instruction was helpful for me.