Eclipse Technical Background

People are central to your company’s performance. With eclipse we present a scientific, validated approach enhancing corporate performance by identifying, changing and aligning your people’s behaviours to help them achieve their best.

Introduction to the Eclipse Behavioural Questionnaire

The Behavioural Questionnaire is the product of more than 25 years research with a clear mission:

To apply a scientific approach to observing, describing, measuring and understanding the behaviours that sustain and move organisations forward, as well as the behaviours that impede growth and success.

The Eclipse Diagnostic Suite

The Behavioural Questionnaire is part of the Eclipse Diagnostic Suite. It has been developed (and continues to be developed) for use in a wide range of situations and we work with each of our clients to determine the most effective solution for their needs. Eclipse is a bespoke system.

The Behavioural Questionnaire is designed to work alone or be used in conjunction with management competency or performance management & development frameworks. It is intended for use as a discussion, feedback, counselling and development tool.

How does the Behavioural Questionnaire work?

The Behavioural Questionnaire employs state-of-the-art electronically based questionnaire survey techniques and is easy to use, generally requiring less than an hour of individual manager time. For all applications, data is initially collected through one of the Behavioural Questionnaire cornerstone diagnostics at the individual level.

The Behavioural Questionnaire asks people both what they think they are actually doing, and what they think they should be doing to be most effective. It can also be used to get input from others on what a person is actually doing and / or should be doing.

Data is analysed, interpreted and reported back in terms of behaviours that people are using and may want to continue to use, as well as those behaviours they may want to avoid. It also shows behaviour gaps - where behaviours that should be used to manage a job effectively are not being applied.

When aggregated, individual data provides the basis of a mapping process which can be tailor-made to client requirements. This can provide an effectiveness snapshot of the organisation as a whole or by team, function, or division.

What the Behavioural Questionnaire does

The Behavioural Questionnaire profiles the 3 key sets of management behaviour that make a difference to corporate success:

those which help an organisation sustain momentum such as:

• Goal setting, performance management, attention to detail
• Training, developing, supporting
• Planning, analysis, application of process and procedure

those which help accelerate momentum such as:

• Taking initiative, inspiring, setting an example
• Delegating, coaching, creating learning
• Coordinating, integrating, acting strategically

those which block momentum such as:

• Showing frustration, annoyance, pressuring
• Avoiding conflict, compromise
• Avoiding responsibility and involvement

These behaviours have been identified through a painstaking process of testing with management populations from many different types and sizes of organisations over a period of 20+ years. They are everyday, observable behaviours based in organisational reality and are expressed in common management language.

By measuring approximately 400 specific management behaviours at individual, team, divisional or organisational level, Momentum CPI provides valuable two-way analysis:

Organisational behaviour mapping

The term "mapping" is not accidental. The Behavioural Questionnaire process shows current and perceived ideal behaviour at every level in a manner that places an individual or group "on the map relative to the position of other individuals or groups". The result is not merely a GPS location system but a road map showing how to change behaviour and move to another position.

Data mining

The Behavioural Questionnaire compares what people are doing with what they think they should be doing. It also shows propensity to change, i.e. how easy change will be. Accumulated data can be analysed from the top-level through to specific detailed behaviours. They can be described, measured and examined from the macro to the micro level.

Research base & validities

The Behavioural Questionnaire has evolved over the lifetime of BSS and incorporates research with over 10,000 managers worldwide across a wide and diverse customer base, including:

• General Motors
• Coopers and Lybrand
• Westinghouse
• Price Waterhouse
• US Air Force
• Nova Gas
• Government of Canada
• Canadian Health System
• Johnson & Johnson
• Rothmans
• Mobil
• P & O
• Xerox
• BP

Reliability and validity of the Behavioural Questionnaire

A study of the reliability and validity of the Behavioural Questionnaire has been conducted at the London School of Economics, with the following extremely positive results:

Reliability

The Behavioural Questionnaire has a reliability of 0.992. The average error is 0.008, or 0.8%

Validity

The Behavioural Questionnaire has a validity of 0.933. The average error is 0.067, or 6.7%.

Survey instrument

The Behavioural Questionnaire consists of 144 pairs of behavioural statements (phrased as "I do X" or "I do Y") and requires the respondent to allocate five (5) points between the two alternatives. Hence, there are six available options: 5-0, 4-1, 3-2, 2-3, 1-4, and 0-5. Choices may be between two favourable or unfavourable alternatives; however the respondent is always required to express preference for one statement over the other. There is no opportunity to allocate equal weighting to both alternatives.

Reliability test

Reliability means 'replicability' and may be measured by whether, and to what extent, responses provided on a test, re-test basis are identical. To assess the reliability of the Questionnaire, 23 managers were asked to complete the Behavioural Questionnaire twice, first on day one and a second time on day six. In order not to influence their responses, they were provided with no background information about the measures or the underlying model. The second Behavioural Questionnaire contained the same statement pairings as the first one but in a different order. This attempted to control for prior response recall, and for direct comparison with prior responses. Participants in the study were managers working for existing clients.

Validity test

In the current context, validity refers to the degree to which the Behavioural Questionnaire measures what it purports to measure. The Behavioural Questionnaire purports to identify and measure the behaviours that managers exhibit in their current jobs. The question therefore is whether the results of the Behavioural Questionnaire - i.e. the reported behaviour - can be verified by other means. This was done by comparing managers' self-assessments of behaviour derived from the Behavioural Questionnaire with objective assessment of their behaviour by observers.

To assess the validity of the Behavioural Questionnaire, 12 pairs of managers were asked to complete the questionnaire. Manager A (the subject) responded to the Behavioural Questionnaire on the basis of how he/she saw him/herself behaving in their current job. Manager B (the observer) completed the Behavioural Questionnaire as he/she saw Manager A behaving in A's job. The paired respondents completed the Behavioural Questionnaire without discussion or consultation with each other. (Note: Observers were selected on the basis of their working proximity with Subjects - i.e. the degree to which they were in a position to actually observe the Subjects' behaviour at work.)

Method of analysis

The data were collated to match up the paired responses to identical questions. For the reliability test the data pairs comprised managers' initial and staggered responses. For the validity test the data pairs consisted of a Subject's and an Observer's responses.

Analysis tested the degree of difference in paired responses. Difference was measured in terms of units - i.e. the difference between a 3-2 response in the first test and a 2-3 response in the second test is one unit, whereas the difference between a response of 3-2 in the first test and 1-4 in the second test is two units difference. The smaller the difference, the greater the reliability or validity.

The formula used to calculate this difference is a modified form of the average deviation equation:

Average error = S [S||diff(a,b)|| / (mpd*q)]/N
where a = first response (reliability test) or subject's response (validity test)
b = second response (reliability test) or observer's response (validity test)
||diff(a,b)|| = absolute value of the difference in degree between a and b - in other words, the unit difference
mpd = maximum possible difference for any one question between the first and second responses (reliability test) or the subject's and observer's
responses (validity test)
q = total number of questions
N = total number of cases/paired respondents

References

Block, J. A. (1957) 'A comparison between ipsative and normative ratings of personality', Journal of Abnormal and Social Psychology, vol. 54, pp. 50-54
Hicks, L. E. (1970) 'Some properties of ipsative, normative and forced-choice normative measures', Psychological Bulletin, vol. 74, no. 3, pp. 167-184