Enterprise Services
Showing results for 
Search instead for 
Do you mean 

Test design contest (Part 1) -- The trend of software test design in Japan

‎04-30-2014 03:50 PM - edited ‎09-30-2015 07:03 AM

By:  Tsuyoshi Yumoto, Test Consultant, Hewlett Packard Japan


Japan map.jpgThe biggest conference for software testing in Japan is JaSST (Japan Symposium on Software Testing) held by the Association of Software Test Engineering (ASTER) which is a NPO that promotes software testing in Japan. JaSST is held eight times a year in various locations throughout Japan. The largest of these conferences was held in Tokyo this March with 1,600 visitors. Conference events included introducing the latest testing trends in Japan, speeches by well-known test researchers and consultants, and the introduction of new published papers.


One of the events at JaSST Tokyo 2014 was the "Test Design Contest - Final Match, 2014” also hosted by ASTER. I was on the panel of judges for this contest and would like to tell you about it in this blog.


In this contest, the competing teams performed test analysis and design using the test basis that the host of the contest provided. Test basis means all of the documents from which the requirements of a system under test can be inferred, and on which the test cases are based. The contest consists of regional matches and a final match. The competing teams submitted artifacts about one month before the regional matches. Regional matches involved evaluation of an analysis method, a design method, and the quality of derived artifacts, and then a presentation. The teams that win the regional matches move onto the final match. The final match also involves evaluation of the derived artifacts and a presentation.


A similar contest was held in the past in which teams competed on the number of defects found in testing, but there were several challenges with this approach. Participants of the contest were unable to see:

  • if the test analysts had the right perspective to analyse the test basis,
  • if the artifacts of test analysis were well arranged for the following phase of test design, and
  • if the test design techniques were applied appropriately based upon the analysis.


At the contest, the judges only focused on the number of defects found by the teams, and not on the artifacts of test design. This time, at the Test Design Contest, artifacts of test analysis and design are the main focus of the judges. This is what makes the contest unique and interesting.


I am not familiar with the culture of other countries, but in Japan, test analysis and test design artifacts are seldom produced in development projects. Engineers who receive test case documents will not know how the test cases were designed and from where they were derived. Test designs then tend to remain within the designer’s head. By showing test designs to each other at the contest, we would like the participants to increase awareness that software testing is an engineering activity which requires an analysis and a design. It also provides an educational opportunity for software testing engineers regarding software test design that can improve their technical skills. These are the overall aims of the contest.


A record high of 27 teams participated in this year’s contest. Seven teams made it through the qualifying rounds and individually presented their test designs to the judges in the final round.


This time, the test basis subject was vending machine software. Each of the seven teams at the final had their own unique ideas; there was a team which designed tests focused on product and testing stakeholders, a team that designed tests by investigating quality characteristics and defects in the past, and a team that started a test design after arranging the functions of the product.Though the test object is the same, there are so many ways to approach it. The quality of the test design of the teams which made it to the final round was of a very high level, and the upward trend of the maturity of test design in Japan was clearly evident.


The artifacts of the teams that made it to the final round are available at this report site.This website is only in Japanese, so I will explain the product features of each team briefly in my next blog.


Related links:

About the Author


Yumoto - cropped.jpgTsuyoshi Yumoto, Test Consultant, Hewlett Packard Japan

Yumoto has been working within IT for over 20 years with a focus on professional testing. He consults with numerous clients on testing process improvements and maturization. He is also a recognised tool expert, having successfully deployed dozens of testing tools such as capture replay, test management, automation and performance tools. He is a member of ASTER and the Japan Software Testing Qualifications Board (JSTQB).

0 Kudos
About the Author


Nov 29 - Dec 1
Discover 2016 London
Learn how to thrive in a world of digital transformation at our biggest event of the year, Discover 2016 London, November 29 - December 1.
Read more
Each Month in 2016
Software Expert Days - 2016
Join us online to talk directly with our Software experts during online Expert Days. Find information here about past, current, and upcoming Expert Da...
Read more
View all