Competitor Analysis with Wappalyzer: A Comprehensive Guide

Competitor Analysis with Wappalyzer: A Comprehensive Guide

What is Wappalyzer?

Wappalyzer is a versatile tool used to gain insights into the web technologies employed by websites. It can identify content management systems (CMS), eCommerce platforms, web servers, JavaScript libraries, and more. One of the most accessible features is its Google Chrome extension, which allows users to inspect a website’s technology stack directly from the browser. This capability is similar to that of BuiltWith, another popular tool for analyzing web technologies. Incidentally, my Python-savvy colleague Greg Bernhardt recently wrote an article about utilizing the BuiltWith API, which might be worth checking out if you’re interested in expanding your knowledge of web technologies.

Why Use Wappalyzer for Competitor Analysis?

Wappalyzer also offers an API, which provides 50 free tokens each month, allowing users to query detailed data about various websites. Each query uses one token and can return a plethora of information, including but not limited to:

• The website’s technology stack (e.g., CMS, frameworks, libraries)
• Social media accounts (Twitter, Facebook, LinkedIn, etc.)
• Contact information (email addresses and phone numbers)
• IP country origin
• Meta titles and descriptions

These insights can be invaluable when analyzing competitors, particularly when trying to understand the tools they use for marketing, tracking, or optimization.

Setting Up: Making Your First Wappalyzer Request

The process of making a request to the Wappalyzer API is straightforward. You’ll need to sign up on Wappalyzer’s website to obtain an API key, which will allow you to start querying websites. Here’s a simple example using Python:

import requests

url = 'https://api.wappalyzer.com/lookup/v2/?urls=example.com&sets=email,phone,contact,social,meta,locale'
headers = {'x-api-key' : 'your_api_key'}
response = requests.get(url, headers=headers)
response_data = response.json()

This code sends a GET request to Wappalyzer’s API to retrieve all available information for a specified website (in this case, example.com). The response will come back in JSON format, and you can extract various data points, such as:

• Technologies used (response_data[0][“technologies”])
• Social media profiles (response_data[0][“twitter”], etc.)
• Language of the website (response_data[0][“language”])
• Meta titles and descriptions (response_data[0][“title”], response_data[0][“description”])

Advanced Data: What Else Can Wappalyzer Provide?

In addition to web technologies, you can also request sets of data that provide contact information, meta details, and social media presence. Here’s how you could make a more comprehensive request:

url = 'https://api.wappalyzer.com/lookup/v2/?urls=example.com&sets=email,phone,contact,social,meta,locale'

The API will still consume only one token, regardless of the amount of data requested. This is useful for competitor analysis, as it allows you to collect all available information in one shot.

Building a Competitor Heatmap

Once you’ve gathered information on multiple competitors, one effective way to compare their technology stacks is through a heatmap. This visualization can help highlight patterns, such as which websites are utilizing common advertising technologies, content management systems, or tracking tools.

Step 1: Collect Data from Multiple Websites

First, you’ll need to iterate over a list of competitor websites:

list_websites = ['example1.com', 'example2.com', 'example3.com']
list_response = []

for site in list_websites:
    url = f'https://api.wappalyzer.com/lookup/v2/?urls={site}&sets=email,phone,contact,social,meta,locale'
    headers = {'x-api-key': 'your_api_key'}
    response = requests.get(url, headers=headers)
    list_response.append(response.json())

This code queries each website, collects the data, and appends it to a list.

Step 2: Analyze Web Technologies

Next, you’ll want to iterate over the collected data to identify which web technologies each competitor is using. Here’s a Python snippet that checks for specific technologies, like advertising and analytics tools:

list_technologies = [["URL", "Twitter Ads", "Facebook Ads", "Outbrain", "Criteo", "Google Adsense", "Google Publisher", "AMP", "Google Optimize"]]

for response in list_response:
    tech_flags = [0] * 8  # Initialize flags for the 8 technologies of interest
    for tech in response[0]["technologies"]:
        if tech["name"] == "Twitter Ads": tech_flags[0] = 1
        if tech["name"] == "Facebook Pixel": tech_flags[1] = 1
        if tech["name"] == "Outbrain": tech_flags[2] = 1
        if tech["name"] == "Criteo": tech_flags[3] = 1
        if tech["name"] == "Google AdSense": tech_flags[4] = 1
        if tech["name"] == "Google Publisher Tag": tech_flags[5] = 1
        if tech["name"] == "AMP": tech_flags[6] = 1
        if tech["name"] == "Google Optimize": tech_flags[7] = 1
    list_technologies.append([response[0]["url"]] + tech_flags)

Step 3: Visualize the Data

To visualize the results, we can convert the list into a DataFrame and create a heatmap using Seaborn:

import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt

df = pd.DataFrame(list_technologies[1:], columns=list_technologies[0])
plt.figure(figsize=(10, 8))
sns.heatmap(df.iloc[:, 1:], annot=True, cmap="Blues", yticklabels=df["URL"])
plt.show()

The resulting heatmap will provide an at-a-glance overview of which competitors are using the most common technologies in your industry. For example, you might discover that most competitors are using Google Optimize for A/B testing, while only a few rely on Criteo for retargeting.

Conclusions

Wappalyzer is a powerful tool for competitive analysis, offering deep insights into the web technologies that power your competitors’ websites. By leveraging its API, you can collect and analyze data efficiently, enabling you to make informed decisions about your own tech stack or marketing strategies.

The ability to visualize the data through heatmaps or other graphs helps identify patterns that might otherwise go unnoticed, giving you a strategic edge. Whether you’re investigating competitors’ tracking systems, marketing tools, or content platforms, Wappalyzer can help streamline your competitive analysis efforts.