Situational Incompetence
Situational Incompetence applies when an otherwise experienced executive is placed in a position of authority or accountability for which they lack experience, training or specific skills. In this new role they are effectively incompetent and incapable of providing reasoned advice, guidance or management.
Situational Incompetence has implication for how leaders are selected for complex tasks requiring specialist domain knowledge and/or technical competence. This theory was developed with specific reference to the field of information systems, but may also apply to other disciplines requiring specific knowledge of unique technology in those domains (eg: science, technology, engineering, medicine, maths and etcetera).
The Research
Information Technology projects fail, and the cost of these failures is staggering. This concern has been highlighted and repeated for more than forty years. Yet, recent publications by the Standish Group (2017) have found that where ‘development projects exceed $100 million in labor costs, only 2% are successful, meaning on-time and within budget. Another 51% are considered challenged or over budget, behind schedule or didn't meet user expectations. The rest, 47%, are seen as outright failures'.
Two questions arise regarding IT project failure research. First, why is the success rate of IT projects so poor? And secondly, why, despite the efforts of many, the situation fails to improve? This problem is known as ‘Cobb’s Paradox’. Cobb’s Paradox states: ‘We know why projects fail; we know how to prevent their failure — so why do they still fail?’. Cobb made the observation in 1995 while attend-ing a presentation by the Standish Group (authors of the Chaos series of reports) while working at the Secretariat of the Treasury Board of Canada. Cobb’s observation that “we know why projects fail” should not be taken in a literal, completely black and white sense, rather it should be considered to be a reference to the collective body of expert commentary, opinion, research and project practitioners that have offered solutions.
Cobb was not alone in observing that there is a great deal studied and written about project failure, and that consulting firms propose methodologies and remedies but little actual progress appears to have been made. The International Federation for Information Processing (IFIP) Working Party 8.6 ran a conference to address this specific issue asking ‘why our scholarship has not been more effective. Is the fault one of theory and inadequate understanding? Or is the problem one of knowledge transfer, the failure to embed research knowledge in the working practices of managers and policy-makers’.
The Case Study
The theory of "Situational Incompetence" was developed following an examination of the failed information systems project at Queensland Health to implement an HR and Payroll Solution between 2007 and 2010. The project was deemed by the Queensland Government to be such a significant failure that a Commission of Inquiry was formed to investigate the project and its failings.
The findings of the Commission of Inquiry state that ‘Its (Queensland Health payroll) failure, attended by enormous cost, damage to government and impact on workforce, may be the most spectacular example of all the unsuccessful attempts to impose a uniform solution on a highly complicated and individualised agency’.
The Commissionors' findings were that there exist two primary causes for the failure of the payroll project:
# unwarranted urgency, and
# a lack of diligence on behalf of state officials
The Commissions Report elaborated further on lack of diligence, describing it as ‘poor decisions made in scoping the Interim Solution, in their Governance of the project, and in failing to hold IBM to account’.
The Findings
The theme that was the most consistent throughout the project was that senior management was repeatably made aware of project risks and failings. Reports had been written about the whole-of-government project prior to the creation of the Queensland Health project that specifically enumerated the challenges and risks that needed to be kept front of mind to the QH project team. The prior research literature provided no plausible explanation to describe the fact that senior executives responsible for the direct execution of the project, and departmental executives with governance and oversight accountability apparently ignored all of the advice that they were presented with.
What emerged from the data was that the executives in charge of the project, those executives that operated above the hands-on technical level, were manifestly incompetent when it came to issues of information systems project management. The executives simply did not understand the information that was being presented to them, and interpreted professional concerns raised by Queensland Health team members as “personality conflicts”. These executives were presented with several formal reports outlining risks and issues, and acted in a manner that, under conventional wisdom, would defy rational explanation - the witness statements and project documents provide no evidence of any action being taken to address the issues raised. On more than one occasion IBM complained that employees of Queensland Health were trying to hold IBM to its contract and make IBM meet its obligations. IBM convinced senior departmental management that these staff were interfering in the project and senior management subsequently ordered their removal from the project.
Engelbrecht et al (2017) suggest that inexperienced managers will seek advice and guidance from inappropriate sources.
Kruger and Dunning (2009) offer the observation that the unskilled and unaware are incapable of identifying their own failings, incapable of independently observing and learning from the competence of others, and incapable of identifying competence in others.
These findings have led this researcher to postulate a new theory: Situational Incompetence.
Situational Incompetence applies when an otherwise experienced executive is placed in a position of authority or accountability for which they lack experience, training or specific skills. In this new role they are effectively incompetent and incapable of providing reasoned advice, guidance or management.
Situational Incompetence has implication for how leaders are selected for complex tasks requiring specialist IT domain knowledge and technical competence, it may also apply to other disciplines requiring specific knowledge of unique technology in those domains (eg: science, technology, engineering, medicine, and maths)
The Peter Principle
The Peter Principle: Why Things Always Go Wrong was written in 1969 by Laurence J. Peter and Raymond Hull. The Peter Principle is summarised as ‘Every new member in a hierarchical organisation climbs the hierarchy until he/she reaches his/her level of maximum incompetence’ The essence of the Peter Principle is that as individuals receive promotion based on their performance at one level in the hierarchy, and as they ascend to ever higher levels, the skills and competence required to succeed at those new levels is different to the skills they exhibited and for which they were promoted. Ultimately an individual reaches a level in the hierarchy where they are no longer competent to perform the task and they cease to be promoted. Peter and Hull (1969, p.173) observed that ‘work is accomplished by those employees who have not yet reached their level of incompetence’.
In critique of the Peter Principle, Lazear (2004, p.142) argues that a lower performance after promotion is to be expected, and firms should account for this in their hierarchical strategies and make allowance for new promotees where a ‘decline in ability that is seen after promotion is the natural outcome of a statistical process that displays regression to the mean’. In any event, it is clear that much research supports the notion that when an individual is moved to a role that requires new and unfamiliar skills their performance will decline (at best) and be seen as incompetent.
While The Peter Principle has applicability to the notion of Situational Incompetence, this new theory refers to the need for specific competence in technical domains in order for the experienced executive to be capable of comprehending advice and responding to event stimuli. Where Lazear argues that a performance dip is inevitable with every promotion while the executive settles into their new accountabilities, an IS project is not a promotion within the hierarchy and "settling into the role" is not an option.
Expert Leadership
Goodall examined “expert leadership” where her research shows that 'organizations on average perform better when they are led by individuals with expert knowledge. In random samples of 35,000 US and UK workers, we found that a boss's technical competence is the single strongest predictor of a worker's job satisfaction. Our latest research suggests that this happens because expert leaders understand how to create the right work environment, and how to appropriately assess, consult and support staff’.
Goodall (2006) studied the correlation between University rankings and the citations of University presidents (the most highly ranked leader of a University irrespective of local title). A list of the world’s Top 100 Universities according to Institute of Higher Education at Shanghai Jiao Tong University in their Academic Ranking of World Universities was used as the basis for identifying the candidate institutions of higher education. Data was then collected on the citations of the Presidents (Vice-Chancellors) of the Universities. Goodall’s findings state that “the most highly ranked universities have leaders who are more highly cited” (Goodall 2006 p.9). The research discussed ‘outliers’ and noted that “there are top universities led by presidents with few or no citations. However, these cases are in a minority” (Goodall 2006 p.11). There was a clear correlation between the research expertise and reputation of the University President and the global ranking of that institution.
Goodall, Kahn et al (2011) showed that the competence and technical abilities of the leader are strongly correlated with positive outcomes in basketball, Formula One motor racing, hospital administration, and university administration (amongst others) and that this may be generalisable to other domains.
Expert Leadership is the flip-side of Situational Incompetence, and goes significantly further than the Peter Principle to consider what attributes are required for the successful leadership and governance of complex organisations and activities.
Situational Incompetence has implication for how leaders are selected for complex tasks requiring specialist domain knowledge and/or technical competence. This theory was developed with specific reference to the field of information systems, but may also apply to other disciplines requiring specific knowledge of unique technology in those domains (eg: science, technology, engineering, medicine, maths and etcetera).
The Research
Information Technology projects fail, and the cost of these failures is staggering. This concern has been highlighted and repeated for more than forty years. Yet, recent publications by the Standish Group (2017) have found that where ‘development projects exceed $100 million in labor costs, only 2% are successful, meaning on-time and within budget. Another 51% are considered challenged or over budget, behind schedule or didn't meet user expectations. The rest, 47%, are seen as outright failures'.
Two questions arise regarding IT project failure research. First, why is the success rate of IT projects so poor? And secondly, why, despite the efforts of many, the situation fails to improve? This problem is known as ‘Cobb’s Paradox’. Cobb’s Paradox states: ‘We know why projects fail; we know how to prevent their failure — so why do they still fail?’. Cobb made the observation in 1995 while attend-ing a presentation by the Standish Group (authors of the Chaos series of reports) while working at the Secretariat of the Treasury Board of Canada. Cobb’s observation that “we know why projects fail” should not be taken in a literal, completely black and white sense, rather it should be considered to be a reference to the collective body of expert commentary, opinion, research and project practitioners that have offered solutions.
Cobb was not alone in observing that there is a great deal studied and written about project failure, and that consulting firms propose methodologies and remedies but little actual progress appears to have been made. The International Federation for Information Processing (IFIP) Working Party 8.6 ran a conference to address this specific issue asking ‘why our scholarship has not been more effective. Is the fault one of theory and inadequate understanding? Or is the problem one of knowledge transfer, the failure to embed research knowledge in the working practices of managers and policy-makers’.
The Case Study
The theory of "Situational Incompetence" was developed following an examination of the failed information systems project at Queensland Health to implement an HR and Payroll Solution between 2007 and 2010. The project was deemed by the Queensland Government to be such a significant failure that a Commission of Inquiry was formed to investigate the project and its failings.
The findings of the Commission of Inquiry state that ‘Its (Queensland Health payroll) failure, attended by enormous cost, damage to government and impact on workforce, may be the most spectacular example of all the unsuccessful attempts to impose a uniform solution on a highly complicated and individualised agency’.
The Commissionors' findings were that there exist two primary causes for the failure of the payroll project:
# unwarranted urgency, and
# a lack of diligence on behalf of state officials
The Commissions Report elaborated further on lack of diligence, describing it as ‘poor decisions made in scoping the Interim Solution, in their Governance of the project, and in failing to hold IBM to account’.
The Findings
The theme that was the most consistent throughout the project was that senior management was repeatably made aware of project risks and failings. Reports had been written about the whole-of-government project prior to the creation of the Queensland Health project that specifically enumerated the challenges and risks that needed to be kept front of mind to the QH project team. The prior research literature provided no plausible explanation to describe the fact that senior executives responsible for the direct execution of the project, and departmental executives with governance and oversight accountability apparently ignored all of the advice that they were presented with.
What emerged from the data was that the executives in charge of the project, those executives that operated above the hands-on technical level, were manifestly incompetent when it came to issues of information systems project management. The executives simply did not understand the information that was being presented to them, and interpreted professional concerns raised by Queensland Health team members as “personality conflicts”. These executives were presented with several formal reports outlining risks and issues, and acted in a manner that, under conventional wisdom, would defy rational explanation - the witness statements and project documents provide no evidence of any action being taken to address the issues raised. On more than one occasion IBM complained that employees of Queensland Health were trying to hold IBM to its contract and make IBM meet its obligations. IBM convinced senior departmental management that these staff were interfering in the project and senior management subsequently ordered their removal from the project.
Engelbrecht et al (2017) suggest that inexperienced managers will seek advice and guidance from inappropriate sources.
Kruger and Dunning (2009) offer the observation that the unskilled and unaware are incapable of identifying their own failings, incapable of independently observing and learning from the competence of others, and incapable of identifying competence in others.
These findings have led this researcher to postulate a new theory: Situational Incompetence.
Situational Incompetence applies when an otherwise experienced executive is placed in a position of authority or accountability for which they lack experience, training or specific skills. In this new role they are effectively incompetent and incapable of providing reasoned advice, guidance or management.
Situational Incompetence has implication for how leaders are selected for complex tasks requiring specialist IT domain knowledge and technical competence, it may also apply to other disciplines requiring specific knowledge of unique technology in those domains (eg: science, technology, engineering, medicine, and maths)
The Peter Principle
The Peter Principle: Why Things Always Go Wrong was written in 1969 by Laurence J. Peter and Raymond Hull. The Peter Principle is summarised as ‘Every new member in a hierarchical organisation climbs the hierarchy until he/she reaches his/her level of maximum incompetence’ The essence of the Peter Principle is that as individuals receive promotion based on their performance at one level in the hierarchy, and as they ascend to ever higher levels, the skills and competence required to succeed at those new levels is different to the skills they exhibited and for which they were promoted. Ultimately an individual reaches a level in the hierarchy where they are no longer competent to perform the task and they cease to be promoted. Peter and Hull (1969, p.173) observed that ‘work is accomplished by those employees who have not yet reached their level of incompetence’.
In critique of the Peter Principle, Lazear (2004, p.142) argues that a lower performance after promotion is to be expected, and firms should account for this in their hierarchical strategies and make allowance for new promotees where a ‘decline in ability that is seen after promotion is the natural outcome of a statistical process that displays regression to the mean’. In any event, it is clear that much research supports the notion that when an individual is moved to a role that requires new and unfamiliar skills their performance will decline (at best) and be seen as incompetent.
While The Peter Principle has applicability to the notion of Situational Incompetence, this new theory refers to the need for specific competence in technical domains in order for the experienced executive to be capable of comprehending advice and responding to event stimuli. Where Lazear argues that a performance dip is inevitable with every promotion while the executive settles into their new accountabilities, an IS project is not a promotion within the hierarchy and "settling into the role" is not an option.
Expert Leadership
Goodall examined “expert leadership” where her research shows that 'organizations on average perform better when they are led by individuals with expert knowledge. In random samples of 35,000 US and UK workers, we found that a boss's technical competence is the single strongest predictor of a worker's job satisfaction. Our latest research suggests that this happens because expert leaders understand how to create the right work environment, and how to appropriately assess, consult and support staff’.
Goodall (2006) studied the correlation between University rankings and the citations of University presidents (the most highly ranked leader of a University irrespective of local title). A list of the world’s Top 100 Universities according to Institute of Higher Education at Shanghai Jiao Tong University in their Academic Ranking of World Universities was used as the basis for identifying the candidate institutions of higher education. Data was then collected on the citations of the Presidents (Vice-Chancellors) of the Universities. Goodall’s findings state that “the most highly ranked universities have leaders who are more highly cited” (Goodall 2006 p.9). The research discussed ‘outliers’ and noted that “there are top universities led by presidents with few or no citations. However, these cases are in a minority” (Goodall 2006 p.11). There was a clear correlation between the research expertise and reputation of the University President and the global ranking of that institution.
Goodall, Kahn et al (2011) showed that the competence and technical abilities of the leader are strongly correlated with positive outcomes in basketball, Formula One motor racing, hospital administration, and university administration (amongst others) and that this may be generalisable to other domains.
Expert Leadership is the flip-side of Situational Incompetence, and goes significantly further than the Peter Principle to consider what attributes are required for the successful leadership and governance of complex organisations and activities.
Comments