Design For Six Sigma (DFSS) Approach for Creating CLEAR Lessons Learned Database

Project personnel working in construction sites fail to transfer invaluable experiences gained mostly due to the absence of a formalized process to record such information. Construction projects are seldom repetitive in nature and this highlights the need for organizations to have in place robust data repositories to facilitate knowledge sharing. This paper describes an effort in creating a new internalonly web-based lessons learned database named Communicate Lessons, Exchange Advice, Record (CLEAR) for the North Carolina Department of Transportation (NCDOT). A Design For Six Sigma (DFSS) approach of Identify, Define, Develop, Optimize, and Verify (IDDOV) model was used. Findings from this study will help NCDOT to institutionalize knowledge and improve project cost variations and schedule predictability. In conjunction with this database, a data dashboard is envisioned to provide effective visualizations for the upper management to make informed decisions based on the lessons information in the database. The dashboard will include success metrics such as detecting reduced numbers of claims and claims amounts, witnessing periodical increase in lessons uploaded into the database, and enhanced communication among specialized staff. This approach is a significant contribution to the existing body of knowledge in lessons learned database implementation for construction applications. The anticipated outcome of this new application will be a more efficient and effective public organization through reduced claims, improved designs, and construction workflows, and improved policies and standards. Future researchers can make use of information presented in this paper to build new robust lessons learned systems to improve organizational efficiency.


Introduction
The need to document and institutionalize firsthand knowledge gained by construction personnel has expanded over the past several decades. Construction industry is a strong knowledge-based industry which relies heavily on knowledge input by various participants in a project team (Carrillo and Anumba, 2002). Construction project management heavily relies on coordinating teams from all phases of the project lifecycle, e.g. planning, design, construction, and maintenance. Even when precautions are taken, external uncontrolled factors such as utility coordination, right-of-way acquisitions, project funding, and interagency communication can lead to delays and claims (Plotch, 2015). In fact, one in three capital projects risks being delayed, over-budget, and fails to achieve its profit objective. One of the primary reasons organizations repeat their past mistakes is failing to document experiential knowledge (Anderson and Tucker, 1994). Researchers have also looked into using the learning curve effects on project costs (Hajdu, 1993) and durations (Mályusz and Varga, 2016) as an alternative to traditional methods in order to effectively use experiential knowledge. A recent report submitted to the North Carolina Department of Transportation (NCDOT) outlines that on an average, construction claims increase spending by 7 % and schedules pushed by 22 % (Jaselskis et al., 2017).
Lessons Learned databases (LLdb) are an effective means to record and retrieve appropriate information to apprise users about past experiences, both good and bad. Previous research on this topic has explored various approaches for lessons learned experiences in the construction industry (Construction Industry Institute, 2017). Lessons Learned (LL) is one of the 17 best practices recognized by the Construction Industry Institute (CII) for enhanced project performance. The CII report on the topic (Gibson et al., 2008) highlights the three main phases of a lessons learned exercise as collection, analysis, and implementation. The Indiana Department of Transportation (INDOT) was an early adopter using a LLdb. Researchers at Purdue University conducted a series of interviews with personnel at INDOT to improve coordination between the design and construction teams aiming to improve its constructability review program (McCullouch and Patty, 1994). The Kentucky Transportation Cabinet funded a study to develop a constructability lessons learned tool used during the design phase to improve project outcomes (Stamatiadis et al., 2013). NASA has both a Public Lessons Learned System (PLLS) as well as Internal Lessons Learned Information System (ILLIS). The U.S. Army Construction Engineering Research Laboratories (CERL) uses DrChecks, which utilizes client-server architecture for comment sharing among various parties pertaining to design documents over the web.
A joint research by Kentucky Transportation Center and University of Kentucky (Goodrum et al., 2003) surveyed resident engineers, contractors, and consultants to obtain an initial understanding of what they envisioned to be a perfect lessons learned database. The team came up with relevant data fields to be populated in a web-based lessons learned database which could accept files both in text as well as image format. However, this effort did not fulfill its intended purpose as the lessons learned database became defunct once the 2000 rows limit was reached due to untimely action by the authorities. Fong and Yip (2006) assessed the level of readiness of construction professionals in Hong Kong to implement lessons learned systems within their organizations. One of their key research finding was that construction personnel did not prefer to record lessons while the project was ongoing, which can lead to important knowledge being lost. Past researches have also looked into improving organizational learning by creating a web-based lessons learned tool named Learning in Construction Tool (LinCTool) to enter and retrieve information in the form of lessons (Eken et al., 2020). The authors devised three layers of taxonomy to categorize lessons and were able to enter 39 lessons from 11 hypothetical projects. However, the authors opine that it maybe necessary to customize the tool for various organizations since the requirements may vary, specifically keeping in mind stricter data confidentiality and security.
The current research took precautions to prevent the snags from earlier lessons learned efforts and come up with a new robust tool to institutionalize knowledge. This paper describes an effort to design and implement a lessons learned database named Communicate Lessons, Exchange Advice, Record (CLEAR) throughout NCDOT. The proposed lessons learned database will be an internal only web-based database that is mainly intended for personnel associated with all project phases within the construction domain in NCDOT. Personnel from various project phases can appraise about problems that arose in a project and avoid repeating mistakes. For instance, from some of the discussions during data gathering phase, we learnt that currently, no formal process exists for the design team to know if any problems arose from their designs during construction and whether any delays and/or additional monies were involved. The lessons learned database is intended to be used by personnel from all the 14 NCDOT divisions throughout the state as well as the central units. The lessons learned database will provide a platform for interagency communication to revisit past experiences rich in knowledge. This database is also expected to bring in organizational changes to improve processes within the organization. A data dashboard for effective visualization is also envisioned to aid project managers improve project workflow processes and undertake necessary interventions.
A Design For Six-Sigma (DFSS) approach for this project was used which can be considered as novel in this field of research. Previous researches on designing lessons learned databases seem to have overlooked this DFSS approach, although few previous researches may seem akin. A DFSS model allows for robust systems to be designed right from the inception till its design complete. This project makes use of the Identify-Design-Develop-Optimize-Verify (IDDOV) model of DFSS to design this database as discussed in the subsequent section.

Design For Six Sigma methodology
Six Sigma was conceptualized by Motorola in 1986 to reduce defects in their products and improving quality. The sixsigma philosophy is a statistics-based concept that aims at minimizing the number of defects and reducing variation in processes, making it an important aspect for business improvement (Montgomery and Woodall, 2008). Design For Six Sigma (DFSS) utilizes six sigma principles and is a systematic, disciplined problem preventing approach which is widely used to design new robust engineering systems (Alwerfalli and Lash, 2012). The most notable difference between six sigma and DFSS is that while the former is commonly used to reduce variation among existing processes, the latter is applied to new processes. Fig. 1 lists other notable differences between the two principles.
Based on the end-user's requirements in an organization, there exist many models of DFSS that may be used to attain favorable outcomes. The most common models used are I2DOV (Invent, Innovate, Develop, Optimize, Verify), CDOV (Concept, Design, Optimize, Verify), IDDOV (Identify, Define, Develop, Optimize and Verify), DMADV (Define, Measure, Analyze, Design and Verify) to name a few (Hu et al., 2004). The closed-loop IDDOV model best suited the needs of this project to design from the outset and to build an error-free robust lessons learned database since it starts and ends with end-users (Anbari, 2002). Furthermore, the features of the product/process are completely defined by the end-user (Kwak and Anbari, 2006). Six Sigma applications typically involve statistical analysis on processes. However, this research took inspiration from the IDDOV concepts and no statistical measures were adopted as would have been in the manufacturing sector or other related sectors. Fig. 2 shows the five stages of IDDOV model as applied to CLEAR database creation.

DFSS applied to CLEAR database 2.1.1 Identifying end-user requirements
The first phase involved gathering end-user needs from the NCDOT personnel and understanding the features they envisaged as an ideal lessons learned database.
The focus was to learn the current practices of sharing lessons and to obtain detailed information about the end-user's needs. For this purpose, an interview guide was created that contained a list of questions to grasp the respondent's needs. The guide questions were classified into three categories -basic respondent information, current practices, and database requirements. Interview requests were sent to 66 potential respondents at the NCDOT. The potential respondents' information was provided to the research team by the Value Management Unit at NCDOT. By the end of this first step, 32 interviews were conducted with 46 personnel with a total of 813 years of work experience. The interviews were conducted both in-person as well as over the telephone. Inputs from these interviews were well documented to not miss on any important piece of information. By the end of this phase, the research team achieved an accurate idea about current organizational practices to communicate lessons within NCDOT and achieve a proper direction to proceed with designing the intended lessons learned database fields.
The principal stakeholders involved with this database are: • End-users: These are the NCDOT personnel who will enter useful lessons based on knowledge gained in project sites. They will also be able to search for relevant knowledge to understand previous circumstances to avoid repetition of problems. • Gatekeeper: The Value Management Unit at NCDOT will act as the gatekeeper and champion for this database. The responsibilities include checking for completeness of lessons, forwarding lessons to appropriate taskforce members, and subsequently publishing approved lessons certified by the taskforce members. • Taskforce: These are the experts within various disciplines who are responsible to ensure the quality, relevance, and the usefulness of the uploaded lessons. Based on their review of the lessons, the gatekeeper will be informed of a decision to accept or reject the lesson.

Defining the database fields
Simple qualitative analysis such as frequently recurring trends/keywords was performed on the respondent inputs to extract the most relevant information. In addition to this, a risk sheet was prepared that listed possible caveats the end-users anticipated in creating the CLEAR database. Based on these inputs, the research team came up with three segments of user input into the lessons learned database. The first segment solicits basic user information such as name, division, and office information such as email and telephone number. This information is not intended to be displayed while displaying the lesson in appropriate search results, but only for the gatekeeper to contact the uploader in case of any missing/additional information. The second segment inputs information on the issue and solution of the lesson being entered. Users will be able to include attachments such as pictures, pdfs, revised contract language, and other relevant files that make it contextually easy to understand. The third segment records project information the lesson being entered pertains to. A few fields in this segment will be populated from other internally linked databases to expedite data entry process and encourage enhanced participation.

Developing the database
The final database designs were handed over to NC Department of Information Technology (NC DIT) to develop the database. The CLEAR database is housed within the NC Connect portal using SharePoint to display the lessons entry form and MS Access database as its backend. The NC Connect portal covers a wide array of products used by NCDOT personnel for their daily work use and hence was the natural choice to host the lessons learned database. To date, 42 lessons have been received from end-users of which four have been accepted by the gatekeeper and put up for reference. It is an ongoing endeavor to solicit as many high-quality lessons as possible from end-users and the research team is reaching out to additional personnel within the organization to build the lessons repository. The research team also plans to visit a few pilot projects currently being constructed to check for the possibility of collecting more lessons from site engineers, inspectors, resident engineers, and other site personnel. Standard operating procedures for end-users and taskforce members were also developed by the research team that is intended to act as a training resource for first time users of this database.

Optimizing the database
Merriam-Webster defines Optimize as "to make (something) as good or as effective as possible" (Merriam-Webster, 2019). The taskforce members were imparted training on how to use the CLEAR database in-person as well as over video calls and their feedback was collected to understand if they felt any aspect lacking to record lessons. This was done to validate the database designs and its development by subject matter experts and to get their knowledgeable opinion. With regards to space constraints within the database, it will be necessary in the future to archive lessons from an ever-expanding repository. These lessons will primarily pertain to obsolete technologies, effected organizational changes, or other suitable reasons determined by the taskforce.

Verifying for completeness
The final phase of the IDDOV cycle ends with the end-users testing the database and informing the research team about any possible modifications or additions in the database. It is anticipated that the department wide rollout of this database will be complete by early 2020. The suggestions gathered from the end-users will be submitted to NC DIT and necessary rectifications in the database will be made. As mentioned earlier, the IDDOV cycle will continue its five phases till the gatekeeper believes that the database is fully equipped to perform its functions. The gatekeeper would be responsible for ensuring the completeness of submitted lessons and for final uploading of these lessons into the lessons learned database.

Data dashboard
One of the main features of this lesson's learned database is its capability to inform upper management on how the end-users have been utilizing this tool as a knowledge base. The research team will be developing a data dashboard as shown in Fig. 3 that will display essential statistics based on user input. This is a significant contribution to the existing body of work, as most construction lessons learned databases for transportation projects available in the literature utilize just the lessons entry and search functionalities. The information displayed can be seen either as an "Overall", "Division" wise or by "Respondent" inputs as seen in Fig. 3. The top three highest impact lessons based on these three categories displayed can act as an efficient information analyzing tool for the upper management. In addition to this, data pertaining to project characteristics such as project type, project delivery, project phase, and project size for lessons entered will also be displayed. Taking necessary remediation steps based on this displayed information can help bring in organizational changes and improve work processes.
The dashboard above would display information based on numerical data extracted from the input data, although the most important information in the lesson relates to the text entered as lessons. In the past, researchers have attempted to automate the process of extracting information from text using appropriate text mining algorithms in Chinese construction documents (Yu et al., 2008). Although our research team is still in the nascent stages of finalizing an approach to text mine from the lessons learned database, a similar approach by Yu et al. (2008) seems to be plausible with appropriate modifications as may be applicable to this project. Automated data processing methods like neural networks, cluster analysis, genetic algorithms, decision trees and more recently support vector machines have started gaining momentum in the realm of data mining (Zhou et al., 2018). In the future, it is anticipated that this text mining utility will be used for effective visualizations to provide insights into frequently occurring trends within the lessons texts thereby assisting the upper management in taking informed decisions.

Results and future work
Currently, the research team is in the process of collecting lessons to populate the database. To date, the research team has been able to gather and update 42 lessons in all in an ongoing effort to increase the number of useful lessons. The database fields were arrived at based on the initial information gathering phase. The database requirements from the end-users are as summarized below:

Less typing
The respondents looked forward to having more of dropdown menus to select their answers rather than typing it, as it would save some time and effort. This was however challenging for the research team but strived to bring all possible options as a drop-down menu.

Keep data entry simple and less time consuming
Most of the respondents also mentioned that owing to multiple tasks throughout the day, it would not be possible for them to spend more than five minutes on entering/ retrieving lessons. Avoid double data entry Currently, work related details are entered in the engineer diaries as well as HiCAMS -a central NCDOT repository to store information related to contract administration, claims, supplemental agreements. Thus, it was important to devise a strategy so that respondents did not need to enter data multiple times.

Easily searchable Display relevant and specific results
So as to not put off the respondents, it was crucial that the search results displayed were most relevant and suited to the requirements based on the search criteria. There is a risk of non-usage of the database if appropriate search lessons are not displayed which the user is searching for.

Use appropriate filters/tags
To obtain accurate search results, users have an option to narrow down search results based on pre-defined criteria such as date observed, keywords, project type. The more information the user inputs as search condition, the better the search results will be.

Other desired features
Upload images/pdf/email files As they say a picture speaks a thousand words. Being able to upload an image file or other relevant attachments will provide better understanding of the issue and the solution taken to resolve the issue. Provide an impact rating (based on cost and time) Provision to have impact ratings based on the project cost and schedule bearing direct influence on the lesson being entered. A Likert scale was mooted for rating both cost and schedule variations. This rating will be critical to search for highly meaningful impactful lessons.

Create automatic trigger
To complete a lesson learned on projects that experience claims or supplemental agreements above a certain threshold value.

Training
The lessons learned database can act as a training resource for fresh recruits to look up necessary lessons using the search functionality.
During the information and database requirements gathering stage, one of the key issues facing most NDCOT projects pertained to utility relocations/removal and coordination. As a result, the academic research team was requested to summarize all key findings from the interviews pertaining to utilities and submit this information to the Value Management Unit for development of a special utility guide. This was validated through our interviews as well as data on claims on all NCDOT projects that was provided by the Value Management Unit. This gave rise to another research idea looking into the leading indicators of claims arising due to utilities which is currently being performed by the same research team. It is also planned to continue collecting and populating the pilot database and prepare for an organization wide rollout of the database around July 2019. To ascertain the success of the lessons learned database, the success measures would involve increased coordination among design, construction, and maintenance personnel. Additionally, there should be a decrease in the number of claims, supplemental agreements arising in construction projects, in addition to observing reduction in the amounts associated with them.

Discussion
This paper presents a novel method of designing lessons learned databases using Design For Six Sigma (DFSS) approach. An IDDOV model was utilized in creating designs to build a novel NCDOT lessons learned database. Currently there is limited communication among personnel in all the 14 divisions regarding sharing useful knowledge gained in project sites. Compounding to this problem is excessive turnover leading to loss of valuable in-person experience. Thus, there is a need to provide a platform for proper communication among all project lifecycle personnel within the organization. Doing this will help improve design guidelines, best practices, revise specifications and bring to light innovations that have taken place on projects that could be used throughout the NCDOT organization. Effectively sharing information of this nature can reduce project costs and improve project delivery. Utilizing the knowledge repository to revisit stored lessons by personnel from all project phases will be critical for making successful database. For instance, design teams should go through lessons entered by construction and maintenance personnel to understand problems faced in the designs prepared. This will help avoid mistakes that occurred or utilize a good practice that was discovered during construction.
The proposed data dashboard will aid in effective visualization by upper management and in monitoring the status of lessons being entered and utilized for project management. Additionally, the use of text mining and the use of relevant algorithms will help in suggesting the most relevant lessons to users based on keywords and trends. The CLEAR database would facilitate superior design performance, reduce the frequency and impact of change orders, achieve enhanced cooperation, and ultimately accomplish improved operational performance. One of the biggest challenges to overcome will be to maximize the number of high-quality lessons being entered in the database. A sizeable number of lessons will aid in effective application of the text mining algorithms on the docket. A 70-30 distribution between training and test datasets is typical and thus, a significant number of lessons is highly desirable to achieve favorable success rates to rightly train the algorithm. Ultimately, interagency project teams across NCDOT will greatly benefit from utilizing this rich and robust knowledge database. In the long run, the CLEAR database will assist NCDOT to adjust future cost estimates, update standards, and change policies to continuously strive to be an effective and efficient organization to the public.

Conclusions
The lessons learned/best practices database within NCDOT would facilitate better coordination between design teams and construction teams. The overarching goal of creating this database is to achieve superior design performance and thus reduce the frequency and impact of change orders, achieve enhanced cooperation, and ultimately accomplish improved operational performance. Lessons learned are an effective way to document and retrieve wisdom gained from previous projects and apply these learnings to future projects. Project teams are dynamic and seldom repeat themselves in different projects. Also, there are possibilities that the aging workforce will retire before their experiential knowledge can be documented. In either case, there would be a significant amount of wisdom that would be lost if they are not documented in a proper lessons learned/best practice database. This will provide scope for the young generation to take cue and put into practice these lessons learned to realize desired project goals. Project teams across departments at the NCDOT will greatly benefit from utilizing this rich and robust knowledge database. In the long run, the CLEAR database will assist NCDOT to adjust future cost estimates, update standards, and change policies to continuously strive to be an effective and efficient organization to the public.