- Note: The size of the circles reflect the size of the impact of the variable (the difference of the odds ratio from 1); the colour reflects whether the variable is statistically significant or not, and in which direction.
- Note: This illustration displays the actual ordered logit regression results for Q1.
Annual Review 2017/2018
2017 marked the second year in which the BSB Assessment was conducted at member firms. The Assessment asks how far each of the nine characteristics illustrated below is demonstrated within a firm. We would expect a firm that strongly exhibits these characteristics to be better equipped and more likely to serve its customers, clients and members well, than one in which these characteristics are less evident; a hypothesis we will continue to test as we develop our work. We use both quantitative and qualitative data to assess firms against these characteristics.
This second annual exercise enabled us not only to extend our evidence base, but also to explore in greater depth the three themes that emerged from our 2016 Assessment, i.e.:
- the perceived mismatch in many firms between the values espoused by the firm and the way that some employees see business being done;
- helping to develop a culture within the banking sector of responsibility and accountability rather than of blame; and
- identifying practical steps to help promote personal resilience and wellbeing among firms’ employees.
THE BSB ASSESSMENT
The quantitative element of the BSB Assessment consists of an employee Survey asking 37 core questions that allow comparison across and between firms and over time. These may be supplemented in any one year by occasional ‘one-off’, non-benchmarked questions on particular themes. In 2016, 28,122 people responded from the 22 participating firms; in 2017, 36,268 from 25 firms. View Assessment Results.
The qualitative part of the Assessment incorporates views and perspectives from all levels and parts of the firm. These are obtained through a variety of means including (in 2017) written submissions by the firm’s board, interviews with non-executive directors and executives, and focus groups with junior and middle-ranking employees.
All survey data and feedback gathered is presented to firms in a way that preserves the anonymity of survey respondents and the confidentiality and non-attributable nature of all views expressed.
The Survey provides the BSB and its member firms with a unique dataset on organisational culture. As of now, this data relates to only two years, which clearly constrains the conclusions we can or should sensibly draw from it. Nevertheless, this first set of data, relative to the 2016 baseline, allows us to answer or begin exploring several questions. Do the results in 2017 differ from those in 2016, and if so, how? What can we learn about the three themes we identified in 2016? Are there differences that emerge on the basis of, for example, gender or tenure? Are results in some types of firm, or in some parts of the market, different to others? And if we look at the results in both 2016 and 2017, are there any patterns that appear to be emerging?
Before considering the results of the Assessment and the answers to these questions it may, however, be helpful to pause briefly and say something about how we have approached the analysis and presentation of the data this year. A more detailed technical explanation is provided here.
Following this, we will consider the 2017 Survey results and ask what has changed from 2016, and what we can say about the different ways in which different people respond to the Survey questions. Later sections of this Review will then draw on both the Survey and the qualitative findings to inform discussion both of our three key themes and other areas of BSB work during the past year such as consumer outcomes, professionalism and individual accountability.
ANALYSING & INTERPRETING THE DATA
Quantitative (Survey) data
The BSB Survey asks 37 questions. 36 are Likert scale questions, presented as a statement with which respondents are asked to strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree or strongly disagree. The final question is a ‘free text’ question. The questions used in the 2017 Survey can be found here.
The responses to questions can be analysed over time (in this case, 2017 relative to 2016) and with respect to certain demographic attributes and factors, including:
- business area;
- role type;
- location; and
The primary type of analysis we run is an ‘ordered logit regression’1. This allows us to explore how important one attribute is relative to another (the ‘base attribute’) in the way that different questions are answered across the respondent population. Do, for example, women systematically answer a question differently to men once all other factors are controlled for? Do people who are new to a firm respond differently to those who have been with it for several years? The list of attributes that we use in our analysis, and the base attribute against which each is compared, is shown in Figure 1:
As in our last Annual Review, this report illustrates the findings from the Assessment in a number of ways. It provides charts showing the distribution of responses to each of the 36 core ‘Likert scale’ questions and clusters of questions, as well as the range of scores across all participating firms by question and for each of the nine characteristics (available here).
This year, these results are shown for both 2016 and 2017. We also present an analysis of how the demographic attributes described in Figure 1 influence responses. This is done by pictorial representation of the ‘odds ratio’. If, for example, being female rather than being male makes it likely – after controlling for all other factors, such as tenure etc – that a question will be answered more positively, the odds ratio is more than one. If, allowing for all other characteristics, women are likely to respond more negatively to a question than are men, the odds ratio is less than one.
We have depicted these odds ratios using circles of different sizes and colours. Green circles indicate that possessing a particular attribute (e.g. being a line manager) positively influences answers to that question in a statistically significant way, relative to someone characterised by the base attribute (in this example, not being a line manager). Red circles indicate that having that attribute negatively influences the response to the question, relative to having the base attribute. The size of the circle, meanwhile, indicates the strength of the influence of the attribute concerned on the answer.
Fig 1. Base for explanatory variables
|Tenure up to 1 year||Tenure 3 up to 7 years|
|Tenure 1 up to 3 years|
|Tenure 7 up to 15 years|
|Tenure more than 15 years|
|Professional body||Not a member of a professional body|
|Line manager||Not a line manager|
|Customer facing||Non-customer facing|
|East of England|
- An ordered logit model is a regression model suitable when the dependent variable is an ordinal scale, i.e. one where responses can be sorted by a rank order. This is true for our Survey data where questions are answered on a five-point Likert scale from strongly agree to strongly disagree.
- We also include in our regressions variables identifying the firm that a respondent is from, with a large firm as the base. This allows us to control for firm-specific effects. We do not, however, report results for these firm variables.
Qualitative (focus group) data
Analysing qualitative data in a rigorous and consistent way, brings its own challenges. We were, however, keen to see what — if anything — the data from focus groups could tell us about the reasons for the differences between higher-scoring and lower-scoring firms or business areas, on questions relating to our three themes. To help us do this we deployed a technique known as ‘grounded theory’.
Our starting point in this approach was the individual focus group (of which we ran 87 in total in 2017). Each focus group consisted of participants from a single business area of a single firm (e.g. Risk and Compliance in Firm A). We looked at which business areas of firms had, in the Survey, registered the most and least positive responses to particular Survey questions (i.e. ‘higher-scoring business areas’ and ‘lower-scoring business areas’), and where there was a wide divergence of scores, identified the focus groups drawn from these higher and lower-scoring business areas. We then identified the focus groups drawn from the same areas in different firms that were lowest scoring (‘lower-scoring business areas’) and compared the feedback and comments from focus groups drawn from the same areas of different firms at either end of the spectrum.Fig 3. Qualitative analysis: choosing focus groups based on Survey scores
By noting the frequency with which characteristics or practices were mentioned in focus groups from both higher and lower-scoring business areas, we were able to see whether any given practice was strongly associated, associated or not associated with higher Survey scores on a particular question and therefore potentially with good or bad outcomes. Figure 3 illustrates how we chose focus groups from higher and lower-scoring business areas. If, for example, a practice is mentioned by participants in all focus groups from higher-scoring business areas, but not in any focus groups from lower-scoring business areas, this practice may be strongly associated with more positive employee perceptions, and relevant when considering good or effective business practice. Where, however, a practice is mentioned in focus groups from both higher and lower-scoring business areas, it is unlikely to be associated with the difference in the scores (although the practice itself may nevertheless be important, and potentially necessary but not sufficient for a good outcome). Figure 4 illustrates the application of this approach.Fig 4. Identifying practices associated with good environments
In presenting the results of this analysis in this Review and on the BSB website, we have listed the practices mentioned in focus groups and, by each practice, displayed the number of focus groups in which it was mentioned by participants from both higher-scoring business areas and lower-scoring business areas. Higher-scoring firm area focus groups are shown within one circle and lower-scoring within another; each segment in each circle represents a single focus group, with the segment shaded where the practice was mentioned.
It is important to note that this is a sampling approach; it only considers focus groups from the firm business areas with relatively high and low scores on each Survey question relating to our three themes.
THE 2017 SURVEY RESULTS
The responses to each question of the 36 Likert scale questions of the Survey were used to calculate a score (ranging from 0 to 100) against each of the characteristics of our Assessment framework for the firm and for different business areas within it.
As in 2016, the 2017 Assessment found considerable variation in the range of scores across all participating firms for each characteristic. The distribution of the scores within each range also varied. The complete Survey results for 2017 can be found here.
Turning to our analysis of the Survey results, Figure 6 illustrates the results of our ordered logit regressions, which seek to explain how important different attributes are to the way in which questions are answered. The first row of the table shows the 2017 results relative to the base case of 2016. A green circle shows that the score in 2017 is more positive than in 2016, with the size of the circle representing the scale of the change.Fig. 5. Distribution of firm level scores by characteristic
The second and subsequent rows then look across both the 2016 and 2017 data to illustrate the impact that different demographic attributes (such as being female, being new to a firm, working in a particular part of the UK etc) have on the way in which questions are answered. Again, a green circle indicates that possession of a particular attribute is associated with a more positive response than one given by someone with the relevant base attribute with a larger circle indicating a greater difference.
The results shown in Figure 6 are for all of the responses to the BSB Survey, i.e. across all firms and business areas. Banking is, however, a diverse sector in terms of the nature of its activities. Even within a firm, different parts of the business may feel very different to each other, and employees have different perceptions of the organisation as whole.Fig. 6 2016 and 2017 ordered logit regression for all respondents
We now consider in more detail the analytical findings shown in Figure 6; first, the changes over time, and then the influence on responses of different demographic characteristics.
Changes from 2016 to 2017
In our last Annual Review, we set out the results from our first Assessment; a unique data set, but a static one — a snapshot of firms at a point in 2016. With the results of this second Assessment, we have for the first time not only a cross-firm comparison but also a time dimension, and can begin to explore what has (and has not) changed, and in either direction.
Of the 33 Survey questions that are comparable in 2016 and 2017 (see Assessment Results our analysis suggests that, across firms, there has been an improvement in the scores on 25 questions and a decline in none. This result is virtually the same when we exclude from our analysis any firm that did not take part in the Survey in both years.)
While for most questions the size of the improvement is small (even where it is statistically significant), there are some questions where the change is more sizeable. These are particularly so on questions relating to leadership, shared purpose, aspects of respect, and perceptions of the competence and reliability of colleagues.Fig. 7 Regressions and tests for 2017 differences from 2016 (all respondents)
Survey scores are calculated from the percentages of employees who agreed, disagreed or were neutral on each question, and the strength of that agreement or disagreement. Looking at the underlying responses to the questions relating to leadership and shared purpose where improvement was most evident:
- 70% of employees believed in 2017 that senior leaders in their organisation meant what they said, compared with 62% in 2016 — Question 1 (‘Q1’);
- 65% thought that senior leaders took responsibility, especially when things went wrong, compared with 58% in 2016 — Q16; and
- 69% saw no conflict between their firm’s values and the way that business was done, compared with 64% in 2016. The 11% that did see such a conflict was down from 14% the previous year — Q36.
Alongside the improved perception of leaders and a stronger sense of shared purpose, greater proportions of employees said in 2017 that:
- they felt treated with respect (89% compared to 84% in 2016) – Q5;
- their organisation put customers at the centre of business decisions (79% compared to 75% in 2016) – Q9;
- people did not get defensive when their views were challenged (50% compared to 45% in 2016) – Q12;
- people had the skills and knowledge to do their jobs well (84% compared to 80% in 2016) – Q20; and
- their organisation responded effectively to staff feedback (63% compared to 57% in 2016) – Q30.
Improvements in employee perceptions were most evident in Retail and in Commercial Banking. Less sizeable improvements were observable in Functions, while there was almost no year-on-year change in Investment Banking3. This does not, it should be emphasised, mean that the scores in Investment Banking were generally lower than in other areas (on some questions, Investment Banking scores more highly as shown in Figure 5). Nor does it mean that scores of or within the Investment Banking areas of individual firms showed no change over the year. All that we are saying here is that there was in general no overall change in results between 2016 and 2017, from respondents working in Investment Banking.
Overall, the 2017 Survey paints a picture of firms moving in the right direction from a starting point of 2016. We need of course to be clear about the limitations of what we can infer from the information that we have. As it is important to emphasise throughout, two data points do not make a trend. We do not yet know how volatile employee perceptions may be on some of the issues addressed by our questions, and therefore what degree of movement reflects real and sustained change. With only two time periods, we cannot say whether an improvement relative to 2016 is evident also relative to 2015 or earlier years; a particular question score might have been erratically low or high in 2016, with 2017 simply reverting to type. Finally and importantly, changes in the scores reflect changes in the perceptions of banking employees. We cannot be certain that this translates to improvements in customer and societal outcomes, or whether these changes are going far or fast enough.
- The two questions where our models and tests suggest improvement across each of the four business areas are around senior leaders meaning what they say (Q1) and people not getting defensive when their views are challenged (Q12).
Observations across business areas
In general, controlling for all other factors, we observe responses in Retail to be more positive than in other business areas, and in Functions to be less positive than elsewhere (Figure 6). Within Retail the perceptions of branch employees are often significantly more positive than respondents from Business Banking, Private Banking or other areas within Retail. Within Functions, meanwhile, the responses of employees in IT & Operations are particularly negative compared to those of Risk and Compliance or of other functions.
Specific questions where Retail responses are most markedly positive relative to other business areas include respect for Risk and Compliance (Q7), provision of information to customers (Q11), and sharing learning and good practices with others (Q13). The one question where there are significantly more negative responses in Retail than in all other areas relates to whether it is difficult to make career progression without flexing one’s ethical standards (Q4).
While responses in Investment Banking are less positive than in Retail on several questions, in particular Q7 and Q13 (as mentioned) and on work having a negative impact on health and wellbeing (Q29). One some other questions, however, they are more positive, such as customers being at the centre of business decisions (Q9).
Respondents from Commercial Banking meanwhile express relatively negative perceptions around aspects of responsivenessand reliability, and particularly so on internal processes and practices being a concern (Q31).
We explore and comment on the differences and similarities by business area through both our investigation of demographic factors e.g. gender, which we consider next, and in the detailed discussion of themes that then follows.
Returning now to the demographic information set out in Figure 6, some interesting observations can be made as to whether and how responses differ by demographic attribute. As described earlier, the colour of each circle shows whether someone with a particular attribute (e.g. working in an area other than London) is in general more or less positive than someone with the relevant base attribute (in this case, working in London), once all other attributes collected are controlled for. The size of the circle then reflects how pronounced this difference is.
Gender For most of the Survey questions, women were more positive about their firm than men. This was the case in Retail, Commercial Banking and Functions, but not in Investment Banking. Across almost all questions there was no difference in perceptions by gender in Investment Banking. The exception was Q17, with women more likely than men to see people turn a blind eye to inappropriate behaviour. We return to this finding later in the Review (see BSB Work Theme 2: Helping to develop a culture of accountability and responsibility rather than of blame).
Time employed by the firm The second of the demographic attributes shown in Figure 6 and one that appears to influence on responses, is the time a respondent has been with their firm. As context, the distribution of firm tenure among survey respondents is shown in Figure 10. Just over half (55%) of respondents had been at their firm for more than seven years, with tenures longest in Retail Banking.Fig. 10 Tenure distribution by business area 2017
On all questions but one, people in their first year of working at their firm tended to respond more positively than those who had been at their firm for longer. The greatest differences were around believing senior leaders mean what they say (Q1), seeing senior leaders taking responsibility (Q16), being treated with respect (Q5), seeing people try to avoid responsibility in case something goes wrong (Q18), people having the skills and knowledge to do their jobs well (Q20), people delivering on promises (Q25), work having a negative impact on wellbeing (Q29), and firms responding effectively to staff feedback (Q30).
The exception to the pattern was Q19, where people who had been working in their firm for less than a year felt as comfortable challenging a decision made by their line manager as those who had been at their firms longer.
On many questions we observe a pattern where perceptions are most positive at joining a firm (respondents who have a tenure of less than one year), but then fall significantly and remain largely flat, before rising among respondents who had been with their firm for more than 15 years. This uptick among those with the longest tenures is particularly marked in Investment Banking.
A similar pattern has been observed in studies of engagement surveys across a number of sectors, in the UK and elsewhere4. The BSB Survey is not an engagement Survey, and the questions asked are not directly comparable. Our results, are, however, consistent with what the existing body of evidence would suggest about changes in employee engagement over time.
Membership of a professional body Responses from people who said that they were a member of a professional body were in general similar to responses from those who said that they were not.
Having line management responsibility Line management responsibility was one of the two demographic attributes linked to the largest overall differences in Survey responses (the other being tenure at the firm). Those who said that they had line management responsibility were, for almost all questions, more positive about their organisation than those who did not (again, a tendency that will be familiar to analysers of engagement surveys). This difference was often quite marked, most notably with respect to leadership, shared purpose and speaking up. Line managers were marginally more negative on only one question (Q31 ‘Our internal processes and practices are a barrier to our continuous improvement’).
Being in a customer-facing role Respondents who said that they worked in customer-facing roles were more positive than those who said that they did not on Q11 (‘In my organisation people are encouraged to provide customers with information in a way that helps them to make the right decisions’) and Q24 (‘I see the people I work with go the extra mile in order to meet the needs of our customers’). They were, however, marginally more negative on Q9 (‘I believe my organisation puts customers at the centre of business decisions’).
Location Respondents in Northern Ireland, Wales and the West Midlands were in general more positive than those working in London. Respondents in the Channel Islands were by contrast generally more negative in their perceptions. Marked differences by location were observable on four questions, with respondents in London considerably more negative than respondents in just about every other location. These questions were Q7 ‘In my organisation Risk and Compliance are both respected functions’, Q13 ‘In my organisation I am encouraged to share learnings and good practices with others’, Q21 ‘In my role, I am encouraged to continually learn new skills and improve my role-specific knowledge, Q24 ‘I see the people I work with go the extra mile in order to meet the needs of our customers’.
- See e.g.: Mind Gym, Sept 2011, The engaged employee: How to keep your people flourishing whatever the weather, Engage for Success Working Paper; Gemma Robertson-Smith and Carl Markwick, 2009, Employee Engagement: A review of current thinking, Institute for Employment Studies Report 469; and, Dan Harris, Nov 2017, The Tenure Curve: How Tenure impacts Employee Engagement, Quantum Workplace
Informing the BSB’s wider work
Overall, a picture emerges from the 2017 Survey of firms moving in the right direction. The forthcoming 2018 Assessment will help establish the extent to which this change is real and sustained. We are also able to identify differences in perceptions between employees with different demographic characteristics; potentially important for firms when determining how findings are prioritised and acted on.
The results from both the 2016 and 2017 Assessments — both the quantitative findings of the Survey, and the qualitative evidence from focus groups, interviews and boards — are informing the BSB’s work on the three key themes it set out for 2017/18 (the mismatch in some firms between stated values and the way business is done; creating a culture of responsibility and accountability rather than of blame; and the personal resilience and well-being of employees), and on consumer outcomes, professionalism and certification. Each of these areas is explored in the sections that follow, drawing on the 2017 Assessment findings as well as on the BSB’s wider work.