Annual Review 2018/2019

The 2018 Assessment approach

2018 was the third year in which the BSB conducted its Assessment with member firms. The Assessment exercise is designed to provide firms with feedback and information to help them manage culture within their organisations. It does not measure or rank ‘culture’ directly. Each firm will have its own individual culture; some firms may contain several. There is no one good or model culture against which others can be measured and ranked, and no single template or checklist for firms to adopt. Financial services is also no exception to the ongoing change affecting all business sectors and their customers (domestic and global, technological and competitive, social and demographic); a dynamic context that has been a continuous through BSB events and activities over the past year, and captured in particular by Howard Covington, Chairman of the Alan Turing Institute, at a member event in February 2019.

That culture does not lend itself to measurement, however, does not mean that it cannot or should not be assessed and appraised in order to manage it more effectively. In what follows we first describe our Assessment approach and the way in which the results are presented in this Review and provide contextual information on last year’s Survey (e.g. numbers of respondents). We then set out the 2018 Survey results and, drawing on these latest results, explore in more detail some of the thematic work undertaken by the BSB over the last year (on speaking up and listening, values and customer focus, and wellbeing). Finally, this Review highlights areas of work for the coming year arising from the Assessment findings, policy development and broader engagement.

Fig 1. BSB Assessment Framework

Shared Purpose
The BSB Assessment: explaining our approach

The starting point of our Assessment approach is that good organisational cultures are defined not by their type but by their outcomes. Good cultures are those that produce good outcomes for customers, members, clients, employees, investors and the economy and society as a whole. Poor cultures are those that produce poor outcomes. Culture is important, in other words, not for its own sake but because of what results from it.

Measuring outcomes is itself, however, not straightforward. The outcome of a financial services transaction may only be apparent some time – years, even – after the event, and retail customers in particular may lack the information and experience to be able confidently to assess outcomes (as opposed to satisfaction with the service provided, which is also important but not necessarily representative of the wider outcomes picture).

The BSB Assessment addresses this challenge by approaching it from a slightly different direction. It asks how far a firm demonstrates characteristics that we might reasonably expect to be associated with good outcomes for customers, clients etc; characteristics that we would expect to be associated with any good culture in banking (and the absence of which would be likely to denote a poor culture). These nine characteristics, illustrated in figure 1, encompass both ethical and professional attributes. A firm that demonstrates these to a high degree would, we would argue, be better equipped and motivated to serve its customers etc well, than one in which these characteristics were lacking. In applying this framework, we take both a quantitative and a qualitative approach.

The quantitative element consists of an employee Survey which asks respondents to strongly agree, somewhat agree, neither agree nor disagree, somewhat disagree or strongly disagree with 36 statements about their firm; each question relates to one of the Assessment framework’s nine characteristics, and the responses are transformed into a score of between 0 (the negative extreme) and 100. We also ask employees for three words that they would use to describe their firm.

These 37 questions remain the same each year, allowing us to observe differences and change not only between and within firms but over time. Alongside these questions, we may also in any particular year ask a small number of additional, ‘one-off’ questions; these are not used for benchmarking, but which inform our thematic work. In 2018, these additional questions related to speaking up, wellbeing and equality of opportunity by gender.

To support our analysis and help us understand the feedback, we also ask respondents for information on a small number of demographic attributes (e.g. tenure, business area). The anonymity of respondents is of course protected throughout the process, from beginning to end. The questions used in the 2018 Survey can be found here.

Firms can choose to run the Survey on a sample basis (i.e. across a representative sample of their workforce large enough to give statistically valid results) or on a census basis (i.e. across all employees).

The primary type of analysis we run on the Survey results is an ‘ordered logit regression’. This allows us to explore how important one attribute (the ‘variable’) is relative to another (the ‘base attribute’), in explaining how different questions are answered across the respondent population. We might, for example, want to know whether people who are new to a firm — once all other known factors have been allowed for — respond differently to those who have been with the firm for several years. The list of attributes that we use in our analysis, and the base attribute against which each is compared, are shown in figure 2.

The results of this analysis are presented in the form of an ‘odds ratios’. If, for example, being female rather than being male makes it likely – after controlling for all other factors, such as tenure etc — that a question will be answered more positively, the odds ratio will be greater than one. If, allowing for all other characteristics, women are likely to respond more negatively to a question than are men, the odds ratio will be less than one.

Throughout this Annual Review, we illustrate odds ratios using circles of different sizes and colours. Green circles indicate that possessing a particular attribute, e.g. being a line manager, positively influences answers to that question in a statistically significant way (the ‘p-value’) relative to someone with the base attribute (which in this example would be not being a line manager). Red circles, conversely, indicate that having a particular attribute negatively influences the response to that question, relative to having the base attribute.  The darker the shade of each circle, the greater the statistical significance. The size of the circle indicates the strength of the influence of the attribute concerned on the answer.

For the purposes of this analysis negatively phrased questions in the Survey are reversed, so that for every question – whether positively or negatively phrased – a green circle represents a more positive response for the variable attribute relative to the base.

Figure 2 displays the actual ordered logit regression results in 2018 for Q1.

Fig 2. Making sense of the data

The qualitative part of the Assessment uses a variety of approaches to obtain views and perspectives from all levels and parts of the firm. The approaches used may vary from year to year. In 2018 they included written submissions by the boards of participating firms, interviews with non-executive directors and executives, and focus groups with employees. The Assessment section of the BSB website provides further detail on both the quantitative and qualitative components of the BSB Assessment exercise.

Nine of the 26 firms that participated in the Survey in 2018 also chose to take part in the qualitative Assessment exercise. The themes of this year’s exercise were those of our additional Survey questions (speaking up, wellbeing and equality of opportunity by gender) and customer focus. We explored these issues in 89 focus groups covering 837 employees, and in interviews with 71 executives and non-executive directors.

Who responded to the Survey and participated in the Assessment in 2018?

The BSB Survey underpins the largest assessment of behaviour, competence and culture in banking not only in the UK, but globally. It provides the BSB and its member firms with a unique and valuable dataset on organisational culture. 72,024 employees from 26 banks and building societies across the UK responded to the Survey in 2018; this followed 36,268 responses in 2017, and 28,113 in 2016 (figure 3).

We undertook the Survey for the first time in 2018 outside the UK banking sector both with the non-UK banking operations of some member firms and with member-firm non-banking operations in the UK. We also ran the Survey in autumn 2018 for a number of banks in Ireland, providing a first year of baseline data for these firms and for the newly established Irish Banking Culture Board. The numbers in this Annual Review relate only to UK banks and building societies, and not to this wider work. As the latter demonstrates, however, the underlying approach has applicability beyond the UK and beyond the banking sector, and we have been happy to share ideas and experiences across a range of sectors and jurisdictions:

Fig 3. BSB Survey responses 2016 – 2018

The increase in response numbers in 2018 reflects in part the increasing preference among member firms for a census (rather than sample) approach to running the Survey, to give all of their employees the opportunity to take part in the exercise and have their views reflected. Figures 4 and 5 describe the composition of respondents in 2018 by business area, tenure, gender, management responsibility, whether they have a customer-facing role, and location.

The increase in response numbers in 2018 reflects in part the increasing preference among member firms for a census (rather than sample) approach to running the Survey, to give all of their employees the opportunity to take part in the exercise and have their views reflected. Figures 4 and 5 describe the composition of respondents in 2018 by business area, tenure, gender, management responsibility, whether they have a customer-facing role, and location.

Running the Survey over successive years allows the BSB and member firms to see to how results change over time, in both absolute and relative terms. We now have Survey data covering three years, allowing us to analyse change in both 2017 and 2018, and over the 2016 to 2018 period as a whole.

For firms in aggregate, scores across the core 36 Survey questions held largely stable at the aggregate firm level in 2018 with no marked increases or decreases. Figure 6 illustrates the year-on-year change using our ordered logit regressions and green or red circle notation. The first row of the table shows the aggregate 2018 results for firms, relative to those of 2017. Subsequent rows show the results by business area, with improvements continuing to be made in Commercial Banking, and some increases in scores also evident in Investment Banking.

The relatively stable 2018 picture followed a year in which scores either rose or held unchanged across all of the comparable Survey questions. Improvements in 2017 were particularly evident on questions relating to leadership, shared purpose, aspects of respect and perceptions of the competence and reliability of colleagues. By business area, scores improved most notably in Retail and in Commercial Banking, and to a lesser extent in Functions. The changes in 2017 are illustrated in figure 7.

Survey scores in aggregate in 2018 therefore held onto, but did not extend, the improvements of 2017. The picture over the two years taken together is one of improvement overall, but with momentum concentrated in the earlier part of that period.

The results illustrated so far have all related to aggregate scores across firms or business areas.  Even where this aggregate picture shows little change, as in 2018, more movement may be evident between and within firms. The aggregate picture is not necessarily representative of each individual firm in terms of either the level of or direction of change in scores.

While the BSB does not report publicly on individual firms’ Survey scores, we show each year the range of scores and whether and how this is changing. Figure 8 shows the range, distribution and median of firm scores against our nine Assessment characteristics, in 2018 and the preceding two years. More detailed analysis, including the range of results by business area and Survey question, can be found in the Assessment section of this website.

Fig 8. BSB Survey scores by characteristic 2016 – 2018
8. BSB Survey scores by characteristic 2016 – 2018

Figure 9 shows the aggregate score across all participating firms for each of our core 36 Survey questions over the past three years. We can see even more clearly here the general pattern of scores rising in 2017, and these gains then being maintained but not extended in 2018. An exception is Q16, which asks whether senior leaders are seen to take responsibility. The scores here improved markedly in both years.

While these results are unconditional and not controlled for any other factors, the same pattern is evident in the results only of those firms that participated in the Survey in each of the three years, i.e. it is not a function of changes in the set of participating firms during this period.

Figure 10 looks in more detail at Q16 (on senior leaders taking responsibility), which, as well as improving in both 2017 and 2018, registered the largest sustained improvement of any question over the two years taken together. In 2016, 58% of employees said that senior leaders took responsibility, especially if things went wrong. This rose to 65% in 2017 and 66% in 2018; a change accompanied by improvement also in our other leadership-related question (Q1, on believing that senior leaders meaning what they say). This ongoing improvement in leadership-related scores comes in the context of a considerable focus on leadership over recent years, and the introduction of the Senior Managers and Certification Regime.

Fig 10. BSB Survey Q16 ‘I believe senior leaders in my organisation take responsibility, especially if things go wrong’ 2016 – 2018

Having looked at individual Survey questions, we now explore in more detail the extent to which respondents’ demographic characteristics affect the way in which, on aggregate, questions are answered.

Figure 11 uses the same visual approach as figure 2, but presents results now for all 36 of our core Survey questions. It shows that, for example, respondents who are new to a firm are likely to answer more positively than those who have been there for any length of time, and managers to answer more positively than those without line management responsibilities.

Turning next to differences in responses by business area, we can see from figure 11 that, controlling for other factors, respondents in Retail tended to be more positive in their responses in 2018 than employees in other business areas (Retail is the ‘base’ attribute in figure 11; the other business areas that are then compared to Retail show primarily red circles, indicating that their scores were lower than in Retail.) Respondents in Functions, by contrast, tended to be less positive than those elsewhere.

Relative to other business areas, Retail respondents were particularly positive in relative terms about sharing learning and good practices with others (Q13, relating to openness) and innovating in the interest of customers (Q33, relating to responsiveness).

Sustained improvements in Commercial Banking over the past two years mean that scores in this business area are, on a number of questions, close or comparable now to those in Retail. Commercial Banking respondents continue, however, to have relatively negative perceptions about aspects of responsiveness and reliability, and in particular about internal processes and practices (Q31, relating to responsiveness).

Among the other demographic characteristics illustrated in figure 11, we can see that customer-facing employees tended to be slightly more positive than those not in customer-facing roles, though not on questions relating in particular to personal resilience (Q28, which asks about excessive pressure, and Q29, which about the impact of working at the organisation on health and wellbeing). Line managers, as noted previously, tend to respond more positively than employees without line management responsibility, and employees subject to the Senior Managers Regime tend to be more positive than those outside the regime’s scope. Geographic area differences tend on the whole to be small. Northern Ireland showed the most positive responses relative to other areas, and employees in most geographic areas responded more positively than in London to questions relating to sharing learning (Q13), being encouraged to learn (Q21) and going the extra mile for customers (Q24).

Figure 12 shows the scores (uncontrolled for other factors) for each Survey question by business area over the last three years.

Looking at what figure 13 tells us about the link between some of these demographic characteristics and the way that Survey questions are answered, we can see that women responded more positively than men on most Survey questions. This was also the case in 2017 and 2016.

When we look beneath the aggregate firm-level data, however, we find that while this pattern was evidence in Retail, Commercial Banking and Functions, it was not the case in Investment Banking. In Investment Banking, women’s perceptions tended, where there was a difference, to be less positive than those of men. This reflected differences in both women’s and men’s responses. Women in Investment Banking answered our questions somewhat more negatively than women in other business areas, and men in Investment Banking tended to respond slightly more positively than men elsewhere.

A factor that emerges very clearly from figure 11 as having an impact on Survey responses is that of tenure. The perceptions of new employees are, on almost all questions, considerably more positive than those who have been at their firm (or in the sector) for a year or longer. While the BSB Survey is not itself an engagement survey, this observation is consistent with the findings also of many engagement surveys, and chimes with intuition. We would expect, in most cases, someone who has only recently joined a firm to feel positive about that decision, and a relatively new arrival may not yet have had sufficient experience of the firm and the way it works, to offer an informed a view, whether positive or negative.

Respondents tend, in aggregate, to answer the Survey questions less positively as their time with the firm (or in the sector) increases. Among those who have been with their firm or in banking for 15 years or longer, however, scores begin to rise again; a relationship of scores and tenure that looks like a reverse ‘J’ (figure 14). The exception to this pattern is Q19, which asks how comfortable employees feel when challenging their manager. Employees generally feel more comfortable challenging their manager the longer they have been at their firm.

Q37 of the BSB Survey invites respondents to describe their firm in three words. In 2016, the three words most commonly used by Survey respondents to describe their organisations were ‘customer’ (or ‘client’), ‘bureaucratic’ and ‘ethical’, and in 2018, ‘customer’ (or ‘client’), ‘innovative’ and ‘ethical’.  The 25 most commonly used words in each year are shown in figure 15.

Fig 15. Top 25 words employees used to describe their firm 2016 – 2018

By business area, the word most commonly used by respondents across all areas — Retail, Commercial Banking, Investment Banking and Functions — has in all three years of the BSB Survey related to customers or clients (figure 16). That ‘bureaucratic’ has declined overall from the second-most-used word in 2016 to the fifth in 2018, reflects a sharp fall in its resonance among Retail employees. In other business areas, it remains one of the most commonly cited descriptors. ‘Innovative’ has climbed up the scale in all business areas, and particularly in Investment Banking and Functions.

Fig 16. Top 25 words employees used to describe their firm, by business area 2016 – 2018

As well the distribution of words used by employees to describe their firm, we can also look at their overall tone. Are most of the words used, on aggregate, positive or negative, and how does this balance differ between firms and business areas, or change over time?

To help answer this, we took each response to Q37, ‘Use three words to describe your organisation’, and assigned to each word a positive (+1), neutral (0) or negative (-1) score depending on its tone. The resulting net sentiment indicator (or valence) for 2018 and the previous two years of the Survey are shown in figure 17. At a firm level, aggregate net sentiment improved materially in 2017 and edged forward very slightly in 2018; the same pattern of improvement followed by consolidation that, as we have seen, emerges from the scores for Q1 to Q36.

By business area, the same pattern of net sentiment is evident in Retail and Functions. In Investment Banking (and to a lesser extent in Commercial Banking) the improvement is more evenly spread across both years. Retail has the highest valence (i.e. the most positive balance of net sentiment) of any business area, and Commercial Banking the least, though Commercial Banking has, however, also shown the greatest improvement in net sentiment over the three years of our Survey, while Functions has shown the least.



Fig 2. Making sense of the data (using Q1 as an example)

  • Fig 3. BSB Survey responses 2016 – 2018

  • Fig 4. BSB Survey 2018 responses, by business area

  • Fig 5. Who responded to the BSB Survey in 2018?

  • Fig 6. BSB Survey results — 2018 change from 2017

  • Fig 7. BSB Survey results — 2017 change from 2016

  • Fig 9. BSB Survey scores by question 2016 – 2018

  • Fig 11. BSB Survey 2018 results, by characteristic

  • Note: Firm-specific effects are controlled for but not shown here
  • Fig 12. BSB Survey scores across years by business area 2016 – 2018

  • Fig 13. BSB Survey 2018 results — women’s responses relative to those of men

  • Fig 14. BSB Survey 2018 scores by tenure in firm and banking sector

  • Fig 17. On balance, how positive are the words used by employees to describe their firm?