Web Performance Management Quick Start Guide
Neustar offers a free 30--‐day trial to our web performance services. If you have not done so, sign up for a free trial today. You can also sign into your Neustar Web Performance Management account on this page.
Neustar’s free trial gives users access to both monitoring and load testing services. Both services are on one platform and the same script can be used for both.
Once you have signed up for a trial, an activation email will be sent to the email registered to the account. Click on the activation link and sign in using your username and password. An account can be shared with multiple users. Each user has a separate login.
Once you log into the account, you will be able to access website monitoring and load testing from the main menu.
Before setting up a monitor for a website or an application, you will need to create a script. This can be done from the monitoring screen. Neustar recommends using Firefox or Chrome with Web Performance Management.
Creating a Monitor
Select “Create New Monitor” from the Main Screen to get started.
Once in the “Create Monitor” screen, you will be able to name the monitor, set up your script, alerts, and select agent locations. On this screen, there is a usage overview. The usage overview shows the current balance, daily usage, this monitor’s daily usage, days until next deposit, next deposit amount and account outlook.
Name Your Monitor
Name your monitor and add a brief description under “Basic Settings.”
Create a Script
There are several ways to create a script:
- Create a simple script with a URL
- Upload a Selenium Script
- Create a script using the Selenium IDE Recorder
Creating a Simple Script
To create a simple script with a URL, click “Create New Script.” The button will turn green indicating that it is available for selection when you place your mouse over it. After clicking on the button, a pop up box will appear where you can enter in the URL.
Type the URL in the box and click “CREATE” as this will validate and save the script.
Your script should be visible in the “Test Script” field. If the script is not visible, click on the arrow in the “Test script” field, the script should be visible in the list. Select it.
Upload a Selenium Script
Because Web Performance Management is built on the Selenium platform, users can import existing Selenium scripts to use for a monitor. To upload a Selenium script, select “Create New Script.”
Select “Upload a Selenium Script” to upload your existing Selenium script.
Browse to find the script to upload. To skip script validation, check the box next to “Bypass Validation” and the click "Save." When the "Bypass Validation" option is selected, the script is only available for use in a load test.
Creating a Script with Selenium IDE Plugin
Use the Selenium IDE plugin for Firefox along with the Neustar Script Recorder to create a script simply by browsing a website. No scripting is required. Just record, playback, edit, and upload scripts directly into your account. The Selenium IDE plugin for Firefox and Neustar Script Recorder can be found in the Scripting menu.
Sample: Two-Step Script
Since the simple script may be insufficient, Neustar has created a two-step script (see below) that can be used use to test Web Performance Management. This script goes to www.neustar.biz and tracks the user as information is entered into the “Search” field. The script can be modified to meet specific user- driven needs.
var driver = test.openBrowser(); var selenium = driver.getSelenium(); var timeout = 60000; selenium.setTimeout(timeout); var tx = test.beginTransaction(); var step = test.beginStep("Step 1"); selenium.open("https://www.neustar.biz/"); selenium.type("id=query", "ddos"); test.endStep(); test.pause(30000); test.beginStep("Step 2"); selenium.click("css=input.search_button"); selenium.waitForPageToLoad(timeout); selenium.verifyTextPresent("documents for"); test.endStep(); test.endTransaction();
Setting an Interval for the Monitor
An interval for each monitor will need to be set up. Options range from 1 to 1440 minutes. The interval is not how often a monitor samples. Instead, the monitoring interval and number of locations determine how frequently a monitor will sample.
For example, a Monitoring Interval of 20 minutes and a selection of 10 Monitoring Locations will result in the monitoring taking a sample about every 2 minutes.
Setting an Alert
The default alerting policy is “3 strikes.” A strike occurs when the monitor encounters an error. When a monitor encounters a strike, the service will immediately attempt to validate the strike. If the error is reproducible an alert is sent. If the strike is not reproduced on the subsequent attempts the monitor will resume the normal monitoring schedule and no alert is sent. The default policy is extremely customizable and can be changed to meet the specific needs of the monitor. To use the default alerting policy, click the arrow for a drop down menu. Select “Default Alerting Policy.”
Users can also create a quick alert identifying who should receive the alert and set the number of strikes before the alert is sent out. To create a quick alert, click “Create New Policy” and a pop-up menu will appear with fields to complete. Enter in the policy name, description, email addresses of those to receive the alert, and number of strikes before email is sent out.
Click the “Create” button after all information has been filled in. The alert is available for selection in the “Alerting Policy” field.
Advanced Alerting Policy
With Neustar’s intelligent alerting, users can do more than just modify the default alerting policy. Users can also create customized scripts to meet specific needs. For example, you can create a script that will post data to a dashboard. To create an advanced policy, select “Create New Policy” and then “Create an Advanced Policy.”
This will open the screen to create an alert policy. You will need to name the script and list the email addresses of those to receive the alert.
Select “Advanced” to enter the scripting screen. Here you can create a custom Alert Policy or modify the default script.
When your script is complete, click “Save.” The script is available for selection in the “Create a Monitor” screen.
Neustar has a set of APIs that can help with creating alerts. The APIs can be found under “Alerting APIs” on the right hand side of the page.
Web Performance Management has more than 100 agent locations. To select the agent locations, place the mouse over the continent where the agent is located and click.
A popup menu will appear with the available locations. Once you have selected your locations, it will appear in the “Selected Locations” below the map.
Activate the Monitor
After a script and an alerting policy have been created, and agent locations have been selected, the monitor is ready. Select “Activate” to turn on the monitor. Select "Save" to save the setting and activate the monitor at a later time. The Monitor can be turned on from the “Create Monitor” screen.
Real User Measurements
Real User Measurements (RUM) provides information about website performance as experienced by actual users. RUM details how long a page takes to load and where time is spent from the moment the user clicks until the page has completed loading.
- Real User
- Web Beacon
In the Analysis tab, you will see time to first byte, Document Object Model (DOM) load time, and total load time. The data can be filtered by country, browser, and connection speed.
In the Time Series tab, the data is graphed by time. The page also displays the Apdex, page hits, and JS errors in this screen.
The Live Feed shows real-time data about where the users visiting the website are located.
A Private Agent allows for the collection of availability and performance information from anywhere inside your infrastructure chain. With Private Agents, users can:
- View performance from an internal perspective
- Quickly identify and resolve root causes of problems
- Deploy at a moment's notice
- Go beyond Neustar's monitoring network, integrating preferred external agents
If you have Private Agents, they can be found in the “Private Locations” tab. The Private Locations tab is visible when setting up a monitor.
After a monitor has been active and gathered data the next step is to review the information collected. To view data from any monitor, do the following:
- Hover the mouse over “Monitoring” from the Main Menu to activate the drop down menu
- Select “Current Monitors” to see a list of monitors you have created
- Check the box of the monitor you would like to see the data displayed in a graph
- Select “Graph”
The data can be filtered by Uptime and Load Time. Data can be grouped by monitor location or step. Multiple monitors, locations, and steps can be displayed in the same graph. In the Monitor field, you can select to add multiple monitors on the same graph for benchmarking.
The Zoom selection changes the time frame of the data. Preset options and customizable time frames are available for selection. Data collected is stored for 365 days for in-depth trending analysis.
Reports can be created with any of the data collected. Report can be automatically generated and sent to stakeholders. To create a report, go to the “Reports” tab and select “Create Report.”
Under "Report Content," select “Add Template.” This is where templates for Reports are created. Hover the mouse over the “Monitor” field and a drop down menu will appear with a list of monitors available. Select the monitor and set the parameters including time frame, load time or uptime, and how to group them by monitor, location, or step.
Once all parameters are selected, name the graph and save it.
Once the graph is named and saved, it will be available for selection in the report. Select the desired graph for the report and specify the following:
- How often to send the report
- Who the recipients are
Be sure to save and name the report before exiting. Under the “Reports” tab, you should see the report with a green light indicating it is ready for distribution.
When downtime for maintenance or other activity, which may cause the monitor to alert, is expected, monitors can be place in a Maintenance Window. These windows prevent users from being falsely alerted or spammed with alert notifications during the maintenance period.
Setting up a Script
Web Performance Management has both monitoring and load testing on the same platform and the same scripts can be used for both services. If you already have a script for monitoring, the same script can be used load testing. If not, a script will need to be created.
Creating a load test is easy. Select “Create a New Load Test” from the Main Menu”.
Setting up a Load Test
The Basic Settings section contains two fields Name and Description. Name is the name of the load test. Tests can have duplicate names. The Name field is required. Description is an optional description of the load test.
The test time and date is where the start time for the load test is configured. By default the “Run as soon as possible” option is selected. Deselecting the checkbox will reveal the options to choose the Start Date and Time. The time presented should be the local time zone for your area.
A test that is selected to run as soon as possible begins the staging process. The process is where the needed components and initial users to begin the test are brought online. The staging process takes 7 minutes to complete, meaning it will take 7 minutes before a load testing user is executing the script(s)
The Test Plan is where the test duration, max users, and slope is defined. The duration is the time in minutes to execute the phase. Max Users is the number of users to provision for that phase of the test. Slope is how to provision the users, either ramp or constant. Tests generally start with a ramp and then hold the user level constant for the specified duration. Neustar strongly urges you to start your test with a ramp. An initial ramp allows you to better determine a website's breaking point.
The Traffic Sources section is where the test location(s) are selected. Customers wanting to test from multiple locations will select the regions from where they would like to test. The Max Users% is the number of users as a percentage to utilize from the selected location. The Test Plan is shown in the Max Users/Phase.
The script selection has two fields, the script to execute and the allocation percentage of the script. Clicking on “Add Script” will create an additional box for script selection and percentage of allocation. The allocation from the previous script will be halved. The allocations can be changed but must add up to 100.
After all information has been entered, click “Schedule Test” to start the test or schedule the test for the desired time.
During and after a load test a variety of reports and information is available to give a detailed analysis of the test. These reports include:
- Test Plan
- Data Throughput
- Response Time
- Transactions and Failures
The test plan shows how many concurrent users were planned when the test was scheduled and how many were provisioned and executed.
Data throughput displays how many bytes were received by transactions completed during each minute of the test, both as a number of bytes as well as broken down by each test script.
Unique failures are captured with a screenshot for RBU scripts. HTML source code, HTTP Headers and, video playback are also available for more detailed analysis. Load tests executing VU Scripts only provide the HTML headers.
The test results show the average response time per minute, summarized for the total transaction and broken down by each step. A site that scales well will have little variance in response time and very few associated errors. Response times that increase as load is applied indicates the site has reached its saturation point.
Transactions and Failures
The results also provide a per-minute summary of the total transactions, the total failures and a breakdown of where the failures occurred. As response time begins to degrade, more errors will begin to take place. Note, a spike of errors at the end of the test is expected. It simply means the final transactions were interrupted because the test was shutting down.