Success factors for CE research projects
|December 6, 2005
|Last year I read some books about project management and was overwhelmed by all the advice I found. Various books provide different tools and methods to make your project a success. Others provide the experience people have made in special situations. Others provide some statistical data about projects.
While all this information can be very helpful in the right situation, it must be overwhelming for unexperienced project managers. This is probably, because none of the books provides advice on what the crucial success factors are, i.e. what to do first and what to do then, while it is impossible to pay respect to all aspects mentioned in literature. Do I have to improve my project plan before I try to improve the team work in my project? Will the project benefit more from improving the definition of goals and requirements or will it benefit more from a better progress control?
While an experienced project manager is often able to answer the above questions, an unexperienced project manager often is not. But how can the knowledge of an experienced project manager be made explicit so that it can be written down and given to the unexperienced? This is the question which I tried to find an answer for during the last months.
The question was translated into the following goals: 1) Find out what the most important success factors for PM are! A literature analysis should be able to answer this. At least it should turn out what experienced project managers find worth writing about. 2) Quantify the impact these factors have on the success of a project in terms of reaching the project goals and finishing the project on time! Here, an analysis of variance (ANOVA) should help to calculate the impact of single factors on the project success. This could be done by confronting experts with virtual project scenarios whose success they should guess based on their experience.3) Put the results of 1) and 2) into a tool that provides some advice to project managers about their projects! The tool should be easy to use, provide feedback within a few minutes and should be interesting for both experienced and unexperienced project managers.
To keep this work manageable, I decided to focus on Concurrent Engineering (CE) research projects (while the resulting tool should be applicable for other projects as well, see also last paragraph).
Literature analysis. A literature analysis was carried out to identify the most important factors for project success as mentioned in scientific and pragmatic papers. 30 different factors have been found in various papers, many of which have been mentioned repeatedly. Some of them have been proven in big studies with data from 100 companies and were collected over many years.
The relevance of the 30 identified factors was checked and some of the factors have been grouped up so that there finally was a list of 13 factors which seemed to have quite an impact on the success of CE research projects.
Quantification of impact. To quantify the impact of these factors, they have been combined into 27 project scenarios. Project one e.g. started with good progress control, strong leadership, but weak team work. All projects had strong and weak factors, so the experts who had to guess the success of such projects had to decide if the strong factors dominate the weak ones or the other way round.
50 experts in CE research projects with altogether more than 400 years of experience in CE research projects estimated the success of each of those project scenarios. This finally allowed to calculate the main effects that each of the factors had on "reaching the project goals" and "finishing the project on time". This was done by carrying out an analysis of variance (ANOVA). This method also calculated the significance of all factors, i.e. the probability that the analysis results have not been a random effect.
Results. 12 out of 13 factors showed a high significance, which means that it was likely by more than 95-99% that the factors had an impact on the project success, according to the experts' opinions. The significant factors are ordered by their importance: Goals and requirements, project progress control, motivation of project team, leadership, competence of team, stakeholder management, risk management, planning, organisation (responsibilities in the project and support by organisations), teamwork, communication, experienced project manager.
All factors were considered to have a negative impact on the project if they were weak (like e.g. a weak progress control) and a positive impact if they were strong (like e.g. a strong leader in the project). Average factors could have either a negative or a positive impact.
The calculations showed that the impact of the factors varies qualitatively as well as quantitatively. Some examples: A good definition of goals and requirements means +0.7 points in the overall project performance and a weak definition means –1.0 points. An experienced project manager on the other hand will add an additional +0.3 points to the project performance while an unexperienced project manager only results in a loss of –0.2 points.
It could be seen that most factors have quite a strong negative impact on the project success if they are weak. In comparison, the positive effect of strong factors is often lower. So one of the major conclusions is: Eliminate weaknesses before you strive for perfection.
Tool. To provide young project managers with this (and more) information, a tool has been created which asks 12 questions to create a profile of any project. The tool then provides three recommendations which seem to have the best impact on that project. It thus only presents the top recommendations, instead of overloading the users with tens of good recommendations and leaving the user with the decision what to do first.
For the "real experts" the table of effects shows the figures that the tool is working with. This table shows how many points the project got and how many points it can gain improving any factor and how much it can lose getting worse in any factor. A short (random) tip at the end concludes the tool giving some of the general "wisdom" that the analysis of data provided.
Conclusion. ANOVA turned out to be a great method for quantifying the implicit knowledge about projects that the interviewed experts had. Although the experts would not have been able to quantify the impact of the 13 factors themselves, the 27 project scenarios and ANOVA allowed them to do this within a few minutes. Some experts have been randomly picked and confronted with the results of the analysis and they all have been quite surprised how well the analysis matched their point of view.
Besides this, ANOVA provided feedback on the quality of the collected data (significance test). So, after this study, a list of 12 significant factors for the success of CE research projects remained as well as some statements on their quantitative impact on project success (reaching the goals and finishing on time) and a tool which provides all the results of this study in an easy way.
Some words on CE research projects: Although this study focussed on CE research projects, the experts who rated the success of the 27 project scenarios have not exclusively been involved in CE research projects. Therefore, all their answers have been biased by other projects they have been involved in. That is why it is very likely that the results are not only valid for CE research projects, but also for similar projects.
I hope you like my approach and especially the results and the tool presented in this paper. If you like try out the "PM consulting tool".
|Submitted by: René Stach