Tutorial #1: Tuesday, March 27, 2012, 9:00-12:30, Seminar Room 3

Empirical Studies in Reverse Engineering and Maintenance: Employing Developers to Evaluate Your Approach and Tool

Massimiliano Di Penta, University of Sannio, Italy

This tutorial will aim at providing practical guidelines on how to conduct empirical evaluations of software maintenance and reengineering approaches and tools. The tutorial will describe how to conduct empirical evaluations carried out involving human subjects (i.e., developers), and then will provide a “by-example” introduction to the main statistical procedures to be used for analyzing empirical study results.

Tutorial aim and scope

When researchers develop novel reverse engineering and maintenance approaches and tools, they should convince potential users that such approaches/tools will actually be helpful in their everyday tasks. For this reason, empirical evaluations - aimed at evaluating the approach/tool performances and at comparing with alternative techniques - are highly desired.
As software development is a highly human-centric activity, assessing an approach based solely on its performances - e.g., precision, recall - is often insufficient. Therefore, it is necessary to conduct empirical studies involving human subjects, to understand the way a developer uses an approach or a tool, and how such an usage experience would be affected by the developer’s skills and by other factors related to the development context. In recent and past years, researchers have devised guidelines on how to conduct experimental assessment for software engineering approaches and tools.
The aim of this tutorial is to provide practical guidelines in conducting empirical studies in software maintenance focusing, in particular, on the evaluation of software maintenance approaches and tools performed by carrying out studies (e.g., controlled experiments) involving human subjects. The intended audience of the tutorial would be (i) researchers, who require to perform empirical evaluations as part of their ongoing research (ii) practitioners, who would like to use solid empirical procedure to select/assess approaches and tools to be adopted.
The tutorial will be organized in two parts. The first part will overview different kinds of empirical studies that can be performed, and will provide insights on how to plan and conduct them, explaining, in particular how it would be possible to experiment software evolution tasks and how to measure the benefit of a treatment on such tasks. Also it will be discussed how to deal with factors that could potentially threaten the validity of the study results. The second part will provide a “learn-by-example” introduction to various kinds of statistical procedures that are suitable for different kinds of study designs.

Biography

Massimiliano Di Penta is associate professor at the University of  Sannio, Department of Engineering, Italy. He was assistant  professor in the same University between 2005 and 2011. He  received his laurea degree in Computer Engineering in 1999 and  his PhD in Computer Engineering in 2003. His research  interests  include software maintenance and evolution, reverse engineering,  empirical software engineering, searchbased  software  engineering  service-centric software engineering. He  is author of  over 140 papers appeared on journals, conferences and workshops. He serves and has served in the organizing and program committees of several conferences such as ICSE, FSE, ASE, ICSM, ICPC, CSMR, GECCO, MSR, SCAM, WCRE, and many others. He has been general chair of SCAM 2010, SSBSE 2010, WSE 2008, general cochair of WCRE 2008, and program co-chair of SSBSE 2009, WCRE 2006 and 2007, IWPSE 2007, WSE 2007, SCAM 2006, STEP 2005, and of other workshops. He is steering committee member of ICPC, SCAM, CSMR, WCRE, IWPSE, and SSBSE. He is in the editorial board of the Empirical Software Engineering Journal edited by Springer, and of the Journal of Software Maintenance and Evolution: Research and Practice, edited by Wiley. Further information are available on his website http://www.rcost.unisannio.it/mdipenta.

Tutorial #2: Tuesday, March 27, 2012, 14:00-17:30, Seminar Room 3

Pragmatic Design Quality Assessment

Radu Marinescu, Universitatea "Politehnica" di Timisoara, Romania

Assessing and improving quality is paramount in every engineering discipline. Software engineering, however, is not considered a classical engineering activity for several reasons, such as intrinsic complexity and lack of rigor. In general, if a software system is delivering the expected functionality, only in few cases people see the need to analyze the internals. This mini-tutorial is aimed to offer a pragmatic approach to analyzing the quality of software systems. On the one hand, it will offer a brief theoretical background on detecting quality problems by using and combining metrics, and by providing visual evidence of the state of affairs in the system. On the other hand, as analyzing real systems requires adequate tool support, the tutorial will offer an overview of the problems that occur in using such tools and provide a practical demonstration of using state-of-the-art tools on a real case study.

Biography

Radu Marinescu is an associate professor at the Politehnica  University of Timisoara, co-founder and head (since 2002) of the  LOOSE Research Group (loose.upt.ro) and co-founder of Intooitus  (intooitus.com), a young spin-off company creating disruptive  quality assessment tools. His research is focused on the areas of  object-oriented software evolution, quality assurance and software  maintenance. He is co-author of the  "Object-Oriented Metrics in  Practice"  (Springer, 2006), which is  a practical guide on how  developers and architects can use metrics to understand, assess  and improve code and design. Following an IBM Eclipse Innovation  Award (2006), he received in 2009 the IBM John W. Backus Award in a worldwide competition for "having done the most to improve programmer productivity". He has published more than 30 papers related to metrics, quality assurance and software evolution and has served in the last years in more than 20 program committees of international conferences. In 2010 he was the General Chair of the International Conference on Software Maintenance (ICSM). Since 2011 he is a member of the editorial board of the Journal of Software Maintenance and Evolution: Research and Practice (Wiley). For the past ten years, Dr. Marinescu has been constantly working as a consultant and trainer for several well-known IT companies including Siemens, Alcatel-Lucent, Telelogic and Nokia. Since 2007, Radu has served constantly as an expert evaluator for the European Commission (EC) on large-scale research projects. He previously gave a tutorial on quality assessment together with Prof. Michele Lanza and Dr. Tudor Girba at ICSE 2008 (with over 25 participants).