Bash script cache warmer ignoring URLs in Magento XML sitemap?

Bash script cache warmer ignoring URLs in Magento XML sitemap?

August 5, 2026 · Illustrated author avatarBy Stefan Bogdanescu
Bash script cache warmer ignoring URLs in Magento XML sitemap?

Warming Magento Cache via XML Sitemap: A Comprehensive Guide

Introduction

Warming your cache is an essential step to ensure optimal performance for your Magento site. In this guide, we’ll walk you through the process of using a bash script to warm up your cache by crawling URLs from your sitemap.xml file.

Understanding the Problem

You’re currently facing two main issues:

  1. The turpentine’s warm-cache.sh script is returning 0 URLs.
  2. Your custom wget and grep script isn’t accessing any of the URLs either.

Let’s dive into each of these challenges to find a solution.

Analyzing the XML Sitemap

First, let’s take a closer look at your sitemap.xml file located at http://easyfarm.ro/sitemap.xml.

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>http://easyfarm.ro/</loc>
    <lastmod>2017-03-18T15:56:14+00:00</lastmod>
    <changefreq>daily</changefreq>
    <priority>0.8</priority>
  </url>
  <!-- More URLs here -->
</urlset>

Notice the structure of the XML file? It contains a urlset element with multiple url children. Each url represents a single page in your Magento site.

Parsing the Sitemap with Bash

To extract the URLs from this sitemap, we need to parse the XML using tools available on Unix-based systems like Linux and macOS. Two common utilities for this purpose are xmllint and xmlstarlet.

Method 1: Using xmllint

The xmllint command-line tool provides a simple way to extract data from an XML file.

xmllint --xpath '/urlset/url/loc' sitemap.xml

This command uses XPath to select all the <loc> elements within the <url> elements of the urlset.

Method 2: Using xmlstarlet

The xmlstarlet tool is another powerful XML manipulation utility.

xmlstarlet sel -t -m '//urlset/url' -v 'loc' -n sitemap.xml

This command selects all <url> elements under the urlset, and prints only their content (-v 'loc').

Building the Cache Warm-up Script

Now that we can extract URLs from the sitemap, let’s create a bash script to warm up the cache using those URLs.

Step 1: Extract URLs

Save the following script as warm-cache.sh in your Magento root directory:

#!/bin/bash

# Get URLs from sitemap.xml
URLS=$(xmllint --xpath '/urlset/url/loc' sitemap.xml)

# Count number of URLs
NUM_URLS=$(echo "$URLS" | wc -l)

echo "Getting $NUM_URLS URLs from sitemap..."

# Warm cache using wget in parallel processes (adjust N based on your server's capacity)
N=4  # Number of parallel processes
wget --quiet --random-wait --no-check-certificate --recursive --no-parent --level=2 -q $URLS

echo "Warming $NUM_URLS URLs using $N processes..."

Make sure to give the script execution permissions:

chmod +x warm-cache.sh

Step 2: Run the Script

Execute the script with the following command:

./warm-cache.sh

This script will extract all URLs from your sitemap.xml file and use wget in parallel processes to crawl those URLs, effectively warming up your Varnish cache.

Conclusion

In this guide, we’ve addressed the challenges of using a bash script to warm up Magento’s cache by crawling URLs from an XML sitemap. By understanding the structure of the XML file, parsing it with tools like xmllint or xmlstarlet, and building a custom cache warming script, you can now efficiently prepare your site for high traffic.

Remember that optimizing caching is just one part of ensuring optimal Magento performance. To learn more about best practices and advanced techniques, consider exploring our comprehensive guide on Magento Performance Optimization.

Happy caching!

Cache Performance SEO Errors

Our network