HostScore Load Testing Methodologies

This methodology guide provides a consistent framework for assessing web hosting services. With such a framework, HostScore is confident in enabling individuals and businesses to make informed decisions supporting their objectives.

The critical tool in our methodology is LoadView by Dotcom-Monitor, a leading service in website performance monitoring. Through its capabilities, we can simulate user interactions, monitor server responses, and produce actionable insights on different hosting services.

Examples of Our Work

Scope of Testing

Web hosting plans are often multifaceted and require attention to various performance metrics and service types. This section outlines the breadth and limitations of our testing approach to ensure clarity on what will be tested, how, and why.

• Types of Web Hosting Services Tested
  • VPS Hosting
  • Cloud Hosting
  • Dedicated Hosting
• Stress Testing Metrics
  • Response Time
  • Error Rate

Testing Tools Overview

LoadView is a sophisticated service and offers a comprehensive suite of monitoring tools. It simulates user interactions and measures website performance under various conditions. Key reasons for selecting LoadView  include:

• Real-Time Analytics: Offers immediate insights into response times and server performance under load.
• Geographical Distribution: Enables testing from multiple locations, reflecting global traffic scenarios.
• Customization: Allows for tailored stress tests to mimic real-world traffic user behavior.

While LoadView is our primary tool, we occasionally complement it with other monitoring and testing tools. These scenarios typically arise when test data is questionable. 

These tools may mirror, provide additional data points, or specialize in certain aspects of performance monitoring not covered by LoadView.

Testing Methodology

Our approach to website load testing aims to simulate real-world scenarios that could affect a website’s performance. This section breaks down the methodology into actionable steps, ensuring that each web host is fairly tested.

Web Host Configuration

In almost all cases, the web hosting accounts we receive will be subject to the most minimal configurations possible. This ensures that test results are not influenced by incidental changes caused by the testing team.

As such, the only setup procedures carried out are as follows:

• Domain name linking and SSL installation
• Default WordPress Installation (Version 6.4.3)
• PHP update (Version 7x or 8x)
• MySQL update (Version 8x) or MariaDB (Version10x or 11x)
• Creation of 7 Standard WordPress Posts
• Permalink changed to “Plain”
• Installation of WooCommerce with 1 product added

LoadView Configuration

LoadView can be configured in various ways. This allows website owners to run tests under various conditions and observe results for performance fine-tuning. 

Each test consists of two key elements:

• Action Definition: The first element establishes what simulated users do on the website, such as loading a page, scrolling, and clicking a link. This allows us to mimic real-world use behavior for the testing process.
• Test Payload: With the actions for each user now defined, we can simulate an increasing number of active users on the website incrementally to observe server performance. 

Load Test Setups

Our tests aim to note the point at which we can reasonably expect hosting plans to sustain traffic. For example, Cloud-hosted VPS plans should easily sustain standard loads of over 2,000 sessions per hour.

If hosts fail at any point during our tests, it will be notable by an increase in the number of errors encountered by the testing tool.

One essential point to remember is that web hosts seldom fail at once in real-world scenarios. Instead, the server is more likely to struggle and serve some requests slowly while failing to serve others.

Our test setups cater for three of the most likely usage scenarios:

Single Page Visit

• These are defined by the atypical bounce user. The simulated user reaches the site and stay on the landing page.
• During the test, we load up to an average of 95 sessions per minute (5,700 sessions per hour).
• In theory, this translates to having 5,700 users per hour visiting one page at the test site.
• The test is then repeated with traffic coming from different region combinations (US, EU, Asia, Middle East, Africa).

Multiple Page Visits

• These are indicative of forum-style sites. In this scenario, simulated users quickly shift between 9 pages on site.
• During the test, we load up to an average of 20 sessions per minute (1,200 sessions per hour).
• In theory, this translates to having 1,200 users accessing 9 pages of our test site every hour.
• The test is then repeated with traffic coming from different region combinations (US, EU, Asia, Middle East, Africa).

eCommerce

• The key objective is to note the site’s performance as users add products to the shopping cart and check out on the payment page. Remember that this simulation consists of several steps.
• During the test, we load up to an average of 6 sessions per minute (360 sessions per hour).
• In theory, this translates to having 360 customers checking out on our test site every hour.
• The test is then repeated with traffic coming from different region combinations (US, EU, Asia, Middle East, Africa).

Testing Regions

Since most websites will cater to users in various geographic locations, performance testing is configured to load virtual users from specific zones. The zones defined are as follows:

End Note

This methodology was devised in concert with expert technical advisory. However, it remains a work in progress as we continue adding more hosts to our performance evaluation. It is far from perfect and will be revised from time to time.

Revision History

Current methodology version: 1.0 (11th April 2024)

References

• Simon, R. B., & Ahuja, L. (2021). Website Monitoring. In Digitising Enterprise in an Information Age (pp. 339-346). CRC Press.
• Jansen, B. J. (2022). Understanding user-web interactions via web analytics. Springer Nature.
• Anwyl-Irvine, A. L., Massonnié, J., Flitton, A., Kirkham, N., & Evershed, J. K. (2020). Gorilla in our midst: An online behavioral experiment builder. Behavior research methods, 52, 388-407.
• Semerádová, T., Weinlich, P., Semerádová, T., & Weinlich, P. (2020). Technical Aspects of Web Design. Website Quality and Shopping Behavior: Quantitative and Qualitative Evidence, 63-90.
• Khder, M. A. (2021). Web scraping or web crawling: State of art, techniques, approaches and application. International Journal of Advances in Soft Computing & Its Applications, 13(3).