How to set up a continuous testing framework using Selenium, Maven, and Jenkins

by Bas Dijkstra  转自 https://learn.techbeacon.com/units/how-set-continuous-testing-framework-using-selenium-maven-jenkins

Last time I talked about the key components of a test automation framework. Now I'm going to provide a step-by-step explanation of how you can construct a test automation framework using those components. I'll also show you how to prepare that framework to support continuous testing. I've used the method below to help many of my clients improve their testing capabilities.

For the purposes of this tutorial, I'll build a testing framework for the Java ecosystem using these popular, well-established tools:

• Maven: A Java software project management tool
• JUnit: A Java unit testing library
• Selenium WebDriver: A library that is used to automate browser interaction
• Jenkins: A build server that can be used to support continuous integration (CI) and continuous testing

This diagram illustrates how all the components will go together:

 

 

The first tool you need to configure is Maven.

Configure Maven

You'll start creating your framework by downloading and installing Maven. Afterwards, confirm that Maven is configured by running

mvn --version

and checking that the output looks similar to this:

 

 

Next, you'll create a new Maven project. This can be done in two ways. The first is to use the following command in the command line (replace the parameter values as you see fit):

mvn archetype:generate -DgroupId=com.mycompany.app -DartifactId=my-app -DarchetypeArtifactId=maven-archetype-quickstart -DinteractiveMode=false

The second is to use an integrated development environment (IDE) that supports Maven, such as Eclipse or IntelliJ IDEA.

Once you've set up the project, the last thing you need to do in Maven is to define the dependencies of your project. These are the libraries that your project will use—in this case JUnit and Selenium. You define these dependencies in the pom.xml file as follows:

<dependencies>
    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
        <version>4.12</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.seleniumhq.selenium</groupId>
        <artifactId>selenium-java</artifactId>
        <version>3.5.0</version>
        <scope>test</scope>
    </dependency>
</dependencies>

This ensures that the next time Maven is invoked, the specified versions of all dependencies defined in the project are downloaded and linked correctly. All other settings related to your project go in the pom.xml as well, but for now, you're all set up.

Create tests in JUnit

If you haven't done so already, it's time to open the Maven project you just created. The tests you're going to write will be stored under the "src/test/java" folder that is automatically created. It's a good idea to create one or more packages under this folder to group your tests (you can forget about the other packages for now). Here's what the folder structure looks like:

 

 

To create your tests, you will rely on the features provided by JUnit. First, under the "tests" package, create a Java class that will contain your tests. Be sure to have the class name end with "Test." This allows Maven to identify it and run the tests inside the class without further configuration:

 

 

A JUnit test is simply a Java method annotated with "@Test." Ideally, it contains exactly one assertion that compares an actual value to a predefined expected value. Here's a very basic example of a good JUnit test:

@Test
public void anExampleTest() {
        
    // Check that the add() method of a calculator returns 4
    // when it is asked to add 2 and 2
    
    Assert.assertEquals(4, calculator.add(2,2));
}

The JUnit library contains many more types of annotations and assertions, some of which you'll see in the next section.

Introducing Selenium WebDriver into your framework

Now that you've created a Maven project and have seen how to add JUnit tests to it, it's time to add another component to your framework: Selenium WebDriver. (I'll simply refer to this as "Selenium" in the remainder of this article.)

Selenium is a library that allows you to automate browser interaction through code, meaning that by using the Selenium API, you can start browsers, navigate to web pages, click links and buttons, type text strings into text fields, and do many other things to simulate how an end user interacts with a web page.

Use Selenium only to write tests that verify whether a sequence of end-user actions on a web page leads to a predefined result. Abusing Selenium to verify logic that is implemented on a lower level in the application, including mechanisms such as calculation algorithms or decision trees, is likely to lead to inefficient automation.

As an example, you can automate some actions around the search engine on Amazon.com. The Selenium API is pretty intuitive, as you can see in the following test:

@Test
public void firstSearchResultIsRelatedTest() {
    
    // This line tells your test where to find the chromedriver, which is the "glue"
    // between Selenium and the Chrome installation on your machine
    System.setProperty("webdriver.chrome.driver", "src/test/resources/drivers/chromedriver.exe");
    
    // Start a new Chrome browser instance and maximize the browser window
    driver = new ChromeDriver();
    driver.manage().window().maximize();
    
    // Navigate to the Amazon.com home page
    driver.get("https://www.amazon.com/");
    
    // Type "Software testing" in the search window
    driver.findElement(By.id("twotabsearchtextbox")).sendKeys("Software Testing");
    
    // Click on the search button
    driver.findElement(By.xpath("//input[@value='Go']")).click();
    
    // Select the first item in the list of search results
    driver.findElement(By.xpath("(//div[@id='resultsCol']//a[contains(@class,'access-detail-page')])[1]")).click();
    
    // Check that the page title contains the term "Software Testing"
    Assert.assertTrue(driver.getTitle().contains("Software Testing"));    

    // Close the browser
    driver.quit();    
}

As you can see, calls to the Selenium API reflect end-user interaction closely, which makes getting started with Selenium easy. Object identification (finding an element on a web page before interacting with it) can be done in a number of ways, including (as you see in the example above) by using the value of the HTML "id" attribute from an object in the DOM, or by an appropriate XPath expression that queries the DOM for a specific element.

Note that no good coding practices around creating Selenium tests have been applied here. I'll explain about to how improve the stability and maintainability of Selenium tests later on.

Now that you have created your first Selenium test, try to run it using Maven. To do this, you can simply run the following command from the project's root folder:

mvn clean test

This will delete (clean) existing compiled code (.class files) as well as existing test results, ensuring that you're running the latest test code, and then run all tests that Maven can identify in the project.

If all goes well, you'll see a Chrome browser being started, followed by the actions and assertions that you specified in the text, after which the browser is closed again. Maven gives you the following output:

 

 

To illustrate what would happen if a test failed, change the assertion so that it checks for the term "Software Development" instead of "Software Testing." If you change your test code accordingly (I'm only showing the line that needs to be changed here)

// Check that the page title contains the term "Software Development"
Assert.assertTrue(driver.getTitle().contains("Software Development"));

and rerun the test using the same Maven command, you now see that your test fails:

 

 

An interesting side effect is that your browser is not closed after the test fails, leaving your desktop cluttered. This is something I'll address later on as well.

Scheduling and running the test from Jenkins

The final step in setting up your initial test framework is to ensure that your tests are run periodically, as part of a larger build process. To do this, use the popular open-source build server Jenkins. After you have installed Jenkins and have made sure that it is running by navigating your browser to the designated URL (since I'm running Jenkins on my own machine here, this is localhost), you should see that it's up and running:

 

 

To run your tests from Jenkins, you need to create a job that will handle this. Select "New Item," give your job an appropriate name, and choose "Freestyle Project." A new, empty job is created.

 

 

Select "Configure," and under "General > Advanced," check the box next to "Use custom workspace" and set the workspace location to the location of your Maven project. This will point Jenkins to the location where your tests reside.

 

 

Under "Build," choose "Add build step" and then "Invoke top-level Maven targets." Here you can specify the Maven targets that need to be executed to run your test, in this case "clean test":

 

 

Also, you want to show the test results in Jenkins. Luckily, Maven provides test results in an XML format that can be interpreted by Jenkins. All you need to do is specify where these reports can be found after the build has finished:

 

 

Now you're set to run your tests through Jenkins. To do this, simply close the configuration screen and click on the "Build Now" button. Note that, this time, you will not see a browser appearing (at least not on a Windows machine) because Jenkins by default is not allowed to interact with the desktop. This does not influence the result of the tests, but if you would like to see some browser action anyway, you can enable this option in the properties of the Jenkins service.

Looking at the console output of your build, you can see that Maven has indeed taken care of running your tests and that everything was successful:

 

 

Making your framework ready for continuous testing

Now that you have created a test case and have it running through your build server, you need to take the following steps in order to get it ready for continuous testing.

First of all, continuous testing implies that you can run your tests continuously (or, more accurately, continually). This means that you need to configure Jenkins to:

• Run your tests periodically, for example every hour
• Run your tests whenever a developer commits code, so that the developer receives feedback on wanted (or unwanted) effects on the overall codebase.

The second option can be achieved by making your tests part of the main build pipeline, where code that is being committed is compiled, submitted to tests (unit, integration, and other types), and then passed through any number of environments, possibly all the way to the production environment.

This requires you to have tests that are repeatable and runnable on demand. This, in turn, requires well-thought-out strategies for:

• Test data management. The test data required for executing a test should be available at all times, or there should be means to make it available rapidly. This gets more complex the more distributed a system is, since test data should be synchronized throughout the application landscape.
• Test environment management. Not only should the application itself be available in the right environment, but the same also holds for all dependencies with which the application interacts during test execution. Techniques such as mocking and service virtualization can massively improve the ability to test often.

As stated earlier, the test code you have written provides room for improvement as well. Here are a couple of frequently used patterns and practices that apply to Selenium tests:

• The fact that browsers are not closed when a test fails should be handled by moving browser creation and destruction into specific setup and teardown routines, respectively. The JUnit framework lets you do that through the @Before, @BeforeClass, @After, and @AfterClass method annotations.
• The code, as you have seen it, mixes test data, object locators, and web page interaction. This does not result in readable or maintainable code. A common practice used for creating better Selenium code is to use the Page Objects pattern.
• No proper error handling is implemented yet, so your test could potentially break for reasons other than a malfunctioning web page. I prefer to create wrapper methods around the Selenium API calls and handle common exceptions in an appropriate manner using those wrapper methods.
• If the standard reporting generated by JUnit and Maven is not enough to quickly analyze any errors that occur during test execution, you might consider implementing an additional reporting mechanism that generates human-readable reporting in HTML format. This also allows you to add screenshots (here the adage "A picture says more than a thousand words" definitely applies). Examples of libraries that make this easy for you include ExtentReports and Allure.

That's the standard setup, but there are others

While this is one general way to create a reusable, extendable test framework that can be used to support your continuous testing efforts, you do have other choices. Every project is different, and no one solution fits all use cases. To ensure success in your automation efforts, determine what you want from a test automation framework first, then add the components and apply the patterns that will help you get there. That said, the setup I've illustrated above has served me well in countless automation projects. Hopefully, it will do the same for you.

Additional resources

If you're a tester who's still trying to learn programming so that you can move up to automation engineer, check out these three free courses that can help you on your journey:

• Programming basics
• Java basics
• Selenium WebDriver course

其他参考文档：

• jenkins+maven+svn实现简单的一键发布： https://cloud.tencent.com/developer/article/1074908