The world is in an unprecedented situation amidst the COVID-19 outbreak, with uncertainty associated with every aspect of our lives, including the research community. Researchers are anxious about their funding and contracts, PhD students worried about finishing their PhD on time and industry cutting back their research plans. At the bottom of the pyramid, PhD students might well be worst hit. While it is great to see UKRI rising to the occasion and extending support to PhD students with up to six months extension, the PhD journey just got harder in these uncertain times. I have tried to pen a few thoughts on getting through a PhD, especially in uncertain times like now, based on my experiences and learning during mine at WMG, University of Warwick, UK.

1. Plan

Your PhD is one phase in your life where you are truly in control of your time. Later on your life will be controlled (to some degree) by your employer. It is best to leverage this precious gift as you are only accountable yourself. However, it can be difficult to see the end at the beginning of the PhD. Having an outline plan makes it easier to define your personal strategy. You can dictate the pace, assign the mini-goals and you yourself decide your research path. However, be ready to review and modify plans periodically. I found mind mapping exercises as a great way to add structure to your thoughts. It helps you to plan by enabling you to identify when to diverge and converge your research.

PhD supervisors should be a big part of your plan. Most supervisors are very busy people and respecting their time is key. When things need reviewing, you need to be disciplined to get work out to supervisors, giving them enough time for critiquing. One approach I used was to regularly send short chunks of work (thesis chapter or even sections) for comments to my supervisors.

2. Pragmatic

At the start of the PhD, every student has the aspiration of changing the world. I remember within a few weeks of starting my PhD, I created a huge mind map of everything I wanted to achieve. I honestly thought that in three years of my PhD, I would solve the “autonomous vehicle safety” question – an area in which the automotive industry has poured in billions of dollars over the last decade and still hasn’t found an answer. And today, here I am, two years after my PhD, starting my UKRI Future Leaders Fellowship focussed on solving the same question (although having a head start as I am building on what I learnt in the last five years).

So it is essential to manage your expectations (not to diverge too much in your research) and do something really in-depth with great rigour. Here the role of the supervisor is also key to enable the initial diverging nature of research but also ensure the students start converging their research at the appropriate juncture.

By the end of the PhD, it was clear to me that a PhD journey was as much about the tangible output of the PhD, as it was about the learning the process of exploring the unknown. I don’t mean to underplay the tangible impact of a PhD, but want to emphasise that the research approaches you learn in the three years of your PhD, you will apply them for the next thirty years of your professional career. In the long run, the story of your journey is more powerful than the destination itself. It was also one of my biggest motivations and something I realised during my time in the German auto industry while working in a team of PhDs at FEV GmbH.

3. Perseverance

It is essential to have constant curiosity and ask questions (no matter if the answer is not what you wanted or expected). My PhD supervisor, Professor Paul Jennings always said to me (which I now say to those who I supervise) – “If you don’t ask, you don’t get”. It is a simple thought but somehow our human nature makes it hard to put to practice.

Another aspect to overcome during your PhD are the “second year blues”. By mid-way in your PhD, the excitement of starting a PhD is long behind you while you don’t even see how the end product of your PhD would look like. Second-year blues are often cured by strong support, encouragement and constructive feedback (mostly by the supervisor but also your co-PhDs and wider friends).

As PhD students sometime we tend to work alone (especially during the finishing phase) or with limited collaboration, which can lead to feeling isolated and lacking in motivation. Speaking with friends, colleagues and supervisors, and sharing a few laughs (in the current situation virtually!) is essential to keep you going.

The finishing phase (last few months) is by far the toughest phase of a PhD. Writing your 70,000 words thesis can be lonely. For me (having previously been in industry) it was especially frustrating as I thought I would rather focus on the impact of the research I have already done, than write my thesis. However, what this phase teaches you is – finish what you started, no matter how tempting the lures of other things might be. It is your perseverance and discipline that will get you past the finish line.

4. Patience

By the time you hit the second year blues, you would have done lots and lots of reading, experiments may be in design phase, or results may not be what you expected. It will be very easy to get impatient. But then again, if you knew all the problems and their answers, you wouldn’t be doing a PhD!

One thing to always remember is that you are not the first to do a PhD and certainly you will not be the last. Like the clichéd saying – “time is the best healer”, in a PhD’s case, “time will teach you how to do research”. One of my biggest learnings has been to embrace uncertainty.

Whenever you take on a new project in academia or industry, there will always be uncertainty associated with it. You can either be patient and embrace it, or get stressed about it. If you chose the former, you would have taken one step towards success…

5. Positivity

Last, but the overarching thing to ensure you enjoy and succeed in your PhD journey is to be “realistically optimistic”. Every experiment you do will not be a success or every publication you submit will not be accepted in a top journal or conference. Failures are an essential part of your PhD, and learning from them will determine your success (not only in your PhD but also beyond).

However, some supervisors can be overly prescriptive with their students. To them I would say, you may think you are helping your students achieve quick results, but in the long run you are doing more harm. Allowing students to fail and learn from it, is the essence of a PhD. As long as you gave it your best shot, it is “absolutely OK” to fail, even that is a contribution to knowledge and one can (and should) be positive about it.

To top it all, your PhD is a piece of research that you understand the best. You know more about the subject that the examiners judging you in your final viva!

I will end by saying I will always cherish the walk from right to left of the Butterworth Hall (Warwick University graduation ceremony), being handed THE piece of paper. The emotion of those 15 seconds made the three years absolutely worth it. I spent a lot of thought and emotion in writing the “acknowledgement” section of my PhD thesis. Sometimes I still tend to read that section, it is my way of reliving each day of my three years of PhD and the emotion of my graduation…

To end in Paul’s words: “PhD is a training in research”, and personally a journey worth having. I am glad I did it and hope you do too, especially in these uncertain times…

----------------------------------------------

Find out more about Siddartha's research here.

In a recent blog for Zenzic UK, I talked about why absolute safety is a myth and that there is a need to create Informed Safety to ensure safe deployment of Connected and Automated Vehicles (CAVs). The concept of Informed Safety is very apt in the current COVID-19 situation. The easing of lockdown rules doesn’t mean that it is absolutely safe to be going around mingling with people. However, it does mean that given we follow a set of rules (and we know why these rules exist), we may be safe. If anyone is guaranteeing your absolute safety in these times, they would be lying to you!

Similarly, the thought of proving or claiming absolute safety for CAVs is unreasonable. Safe deployment not only needs safe technology but also safe use of the technology, which can be achieved by imparting Informed Safety (which prevents misuse and disuse). Informed Safety essentially means the user is aware of what a system can and cannot do. An aspect of Informed Safety involves understanding ‘conditions’ in which the CAV is capable of operating safely. The CAV industry calls these ‘conditions’ — the Operational Design Domain (ODD). For example, the operating conditions (ODD) of a low-speed shuttle could include a city centre or a business park, and not a four-lane highway!

By defining the ODD, one is essentially in-part defining a CAV’s capabilities and limitations. I would like to point out here that defining the ‘limitations’ is more important (than capabilities). Why are limitations so important? Because limitations illustrate the boundaries of the ODD and when the CAV can no longer operate safely. Thus, the role of ODD is especially important for higher levels of automation — SAE level 3 and SAE level 4.

The Motivation

Be it the creation of the safety case for trials (or commercial deployment), or identifying test scenarios for CAVs, defining the Operational Design Domain (ODD) remains the first crucial step of the process. However, over the last six-seven months I have had innumerable discussions (both nationally and internationally) with industry, government, and academic experts about the concept of Operational Design Domain (ODD). On one hand, I feel super happy and on the other, I feel a bit frustrated! I feel happy because finally (after four years of hype) people are talking about ODD (and asking the right questions) which is fundamental to the safety of CAVs. But I feel a bit frustrated because everyone has their own understanding of what is an ODD, to fit their products or pre-conceived notions (a perfect example of their confirmation bias).

After speaking with much of the CAV community, I believe there is a need to address some of this seemingly apparent void in the industry and add clarity to the question: ‘what is an ODD?’

The Definition

Let us start with the original definition of the term ODD. The Society of Automotive Engineers (SAE) in their SAE J3016 (first published in 2014) introduced the term ODD in the context of CAVs. Since then it has been updated twice with the latest update being done jointly with ISO, which is happening as we speak. According to the 2018 version:

SAE J3016 defines ODD as “Operating conditions under which a given driving automation system or feature thereof is specifically designed to function, including, but not limited to, environmental, geographical, and time-of-day restrictions, and/or the requisite presence or absence of certain traffic or roadway characteristics”.

While some might say the definition is too wordy, it still provides what we need to know about an ODD. The key thing to keep in mind here is the words “operating conditions”.

The Confusion

While the definition of the term ODD does exist, in the last months, I have heard a variety of interesting ideas from many organisations about things that an ODD definition should include. Some of the interpretations did surprise me! From probabilities of pedestrians jumping in front of CAVs to the ability of a CAV to do an unprotected right (or left) turn manoeuvre, from ODDs for simulation to saying ODD is the same as test scenario: it is an interesting mix of ideas.

To be honest, ODD is not the first such instance when the industry and research community have been unable to agree on a definition. A classic example of such confusion is another widely used term — ‘scenario’. During my PhD, I created a glossary of terms after reading many international standards. To my surprise, I found eight (!) different definitions of the term scenario in as many standard documents. So much for standardisation…

Coming back to ODD: ODD is a characteristic of the system (a system property). The keywords in the SAE J3016 definition of ODD are ‘operating conditions’. A manufacturer would choose the specification of its CAV’s sensors depending upon its desired operating condition. For example, the sensors required for a motorway chauffeur system operating at 70 miles per hour on motorways, will be very different from the sensors required for a shuttle operating at 12 miles per hour, ferrying passengers in business parks, and which will be different from sensors required for operating in wintry conditions (like the one below).

The thought of defining probabilities of ‘things’ happening (e.g. pedestrian jumping on the road) as a part of an ODD is a bit of an oxymoron. As ODD defines the operating conditions, the choice of sensors on the CAV should not be based on jumping from an assumption that there is a 50% probability of a pedestrian on the road, to a conclusion that only 50% accurate detection is required. That would be an unsafe system!

On the subject of CAV behaviour (e.g. vehicle manoeuvres): The behaviour of the CAV is not a part of the ODD but rather the outcome of the CAV responding to the driving environment (operating conditions) it encounters, based on its performance or technology limitations. However, when we define the maximum allowable speed of a CAV (not instantaneous behaviour), in terms of the performance limitations of the automation, that would be an ODD limitation. On the other hand, when the top speed was a limitation based on tire performance, it would be a vehicle limitation and thus not an ODD constraint.

ODD ≠ Scenarios (be it functional, logical or concrete)

To keep it simple — ODD is not the same as test scenarios.

If there is one message I would like you to take away after reading this blog, it is this. There is a sizeable section of engineers who are starting to use ODD and scenarios interchangeably. There is no doubt that ODD and scenarios are related and they absolutely should be. However, being related doesn’t mean that they are the same. For some reason, this is a fact I am finding the industry is having a hard time accepting. It might possibly be because the industry is getting indoctrinated with the scenario way of thinking (and rightly so when it comes to testing), and thus have a natural tendency to badge everything we do (or talk about) with the tag of ‘scenario’ — (fans of Daniel Kahnemann (like myself) will call it — the system 1 thinking).

This is possibly the biggest source of confusion leading to the concepts of manoeuvres, probabilities, use in simulation etc. within the concept of ODD. Rather these concepts (scenarios, simulation, and probabilities) are derived from ODDs.

So what is an ODD?

ODD essentially defines the operating environment for which a system is designed for. It may also be seen from the perspective of the end-user (e.g. city council authority) as the operating environment in which a system should be able to operate safely. It is essential that there is an overlap between the two perspectives of the ODD, manufacturer (or the system designer) and the end-user, for ensuring the safe deployment of CAVs. More often than not, one of these perspectives is forgotten. However, I must admit that SAE J3016 doesn’t provide this clarity either. Latest intel (from informed sources) suggests that a new SAE document will address this.

One thing we as an industry have to accept is that an ODD definition will never be exhaustive. One can make a list of ‘n’ attributes used for an ODD definition, but there will always exist a ‘n + 1th’ attribute. Like all safety processes, our goal as engineers is to ensure we do our best! And we do (well, mostly!).

One of the first such attempts at defining a standardised way of specifying the ODD has been BSI PAS 1883 and SAE AVSC Lexicon. Inspired by the work carried out by PEGASUS project (Germany), NHTSA, University of Waterloo and WMG’s role in Streetwise project (UK) and OmniCAV project (UK), BSI PAS 1883 specifies a taxonomy for ODD attributes and has three high-level attributes:

• Scenery: non-movable elements of the operating environment
• Environment: weather and other atmospheric conditions
• Dynamic elements: all movable objects and actors in the operating environment

An obvious question would be — why would anything I say, override the pre-conceived notions. Let me address this head-on, and I must make an honest confession here.

ODD as a term was coined by the SAE J3016 document, created by a team of experts on behalf of SAE’s ORAD standards committee (back in 2014). Initially, I too shared a lot of the confusion and it was not until two years back that I got the clarity in my head. I had the good fortune of having access to some of the authors of the original SAE J3016 and discussed the subject of ODD at length with them. In subsequent years, we have worked on the SAE J3016 revisions together, with most recently being in the joint ISO/SAE working group creating ISO 22736 (the ISO version of SAE J3016).

And as I thought I had mastered ODD, in comes the term ODC — (Operational Design Conditions).

Introducing ODC (Operational Design Condition)

In my more recent (last month) discussions, I have been introduced to the concept of Operational Design Condition (ODC) which is being advocated as a superset of ODD. I feel the concept of ODC, helps clear some of the confusion surrounded around ODD. ODC is said to consist of three things:

• ODD
• Subject vehicle (ego vehicle) capabilities (instantaneous)
• Driver capabilities

ODC is a superset of conditions required for the CAV to operate safely. If we don’t have a detailed understanding of your system’s ODD and ODC, we are essentially creating an unsafe system.

Standardisation galore!

As I mentioned earlier, up until seven months back, there wasn’t much of a discussion around ODDs. All of a sudden, not only there are a lot of discussions, even the standardisation landscape has become a crowded field, with various national and international standardisation bodies wanting a piece of the pie (with their own interpretation of ODD and its usage). Be it ISO or BSI, SAE, or UL or even ASAM (standardisation bodies), everyone is interested in standardising something or the other for CAVs and ODDs.

It is absolutely fantastic to see so much interest in standardisation. However, in our recent BSI CAV Standards Advisory Board (SAB) meeting, it was evident that we need to be careful about things we standardise, and not to standardise immature technology or in an area where we (as an industry) still don’t know enough. For example, in some of the forums I have been involved in, we have spent days trying to get experts to understand what is an ODD before we actually start a standardisation project.

While BSI PAS 1883 defines a taxonomy for ODD definition, ISO 34503 adds a high-level definition format for ODD using the taxonomy, and ASAM OpenODD aims to provide a simulation level format for ODD. UL 4600 provides some considerations for ODD definition and SAE AVSC provides a lexicon for the ODD definition. The subject of ODDs and scenarios is large enough for everyone to have their niche areas. One word of caution here is that we need to avoid creating conflicting standards. Someone needs to play the role of coordinating these activities to ensure that there are no conflicts.

The Hope…

With all of the standardisation activities and confusion, what is refreshing to note is that people are starting to accept that there is a lack of understanding. ODD is key to CAV safety, and if we don’t have a common understanding of it, there will always remain a clinch in our logic for CAV safety. The key to the success of the CAV industry is knowledge sharing and collaboration. I personally have had discussions with SAE members, ISO and ASAM. And the great thing about the CAV industry is that we are willing to learn and accept, given logical arguments. With my mission to ensure the safe realisation of the CAV technology, that’s what keeps me going in this industry and in standardisation…

And while there is hope, great things can always happen…

-------------------------------

I serve as the work item leader for the ISO standard (ISO 34503) on Taxonomy for an Operational Design Domain which is building on the work I led for BSI and CCAV on PAS 1883. A big note of thanks to Dr Steven Shladover (UC Berkeley, PATH Program), Ryan Lamm (Southwest Research Institute, TX, USA), Dr Peter Burns (Transport Canada), David Webb (CCAV, HM Government), Kenji Okamura (JSAE, Japan), Dr Edward Griffor (NIST, USA), Dr George Economides (Oxfordshire County Council), Dr Dave Jones (UK’s Met Office), Dr David Bott (WMG) and Professor Paul Jennings (WMG) for their comments on this blog, and for all the discussions we have had over the past many years.

Original article source here.

I am no fan of Lewis Hamilton (due to my Ferrari allegiances), but developed immense respect for him on a personal level when I read about a conversation between Lewis and Toto Wolff (Mercedes Formula 1 boss):

“He (Lewis) once asked (Toto) me: ‘Have you had the active thought that you are white?’ I said: ‘No, I have never thought about it,’ and he said: ‘I need to think about it [my colour] every day because I am being made aware of it.’

As a six-time world champion, Lewis has achieved almost everything there is in the sport. If he is made aware of his colour, every day, imagine what others less privileged than he, feel like or are made to feel like every day. A lot has been said in recent years and (more so) in recent months about diversity; but the question remains, why are under-represented sections of society still forced to deal with the effects of under-representation every day of their lives?

Unfortunately, we live in a society that talks the talk about diversity but rarely walks the walk. Most conversations around diversity have a siloed approach. As a society living in the 21st century, we should be ashamed that we need a campaign like #BlackLivesMatter when by now we should actually have established that Black lives must matter just as much as White lives and all other lives. No one should have to fight to get what is rightfully theirs.

Given that we live in a society which has failed us, symbolism, role models and diversity champions are needed to raise awareness. However, championing diversity without creating a meritocratic environment will reduce that championing to a token gesture and a tick-box exercise, something most organisations are conveniently doing already.

Awards: A Polarising Subject

I volunteer some of my time for an engineering charity. Back in 2014, I floated the idea of a “woman engineer of the year award” to celebrate woman engineers who have reached great heights in their careers despite the societal and organisational hurdles and stereotypes they might have experienced. For me, awards not only recognise contributions but also create role models.

I was a bit naive in thinking that setting up of the award would be a no-brainer for the organisation’s awards committee. Little did I know that a bunch of white men (‘the awards committee’) sitting in a room in central London would decide that creating an award to celebrate women engineers would be discriminatory!

This is even when I got a major British engineering giant to sponsor the award, so money wasn’t even an issue! To my surprise, the more I spoke to engineers (of all genders) in the UK and internationally, the more I realised that this is a polarising topic. Some of the white woman engineers were passionately against the idea of this award. To be fair, they have been fortunate enough to be brought up in families and organisations where bias was limited and people progressed on merit. But my biggest issue with them is that they wrongly (and perhaps naively) assumed that everyone else has been as fortunate as them.

This was six years ago, and to date, the organisation doesn’t have and doesn’t want to have an award for woman engineers!

The Flawed Argument

I was following the recent uproar on Twitter against the Institute of Physics (IoP) following an ignorant blog post by their Chief Executive and the lack of diversity in their honours and awards. I don’t belong to the institute and neither am I familiar with their way of working. But one statement from that blogpost jumped out for me: “nominate more people from under-represented groups for awards”.

At the face of it, the statement makes perfect sense and yes the responsibility lies with us. But let me give you another fact. A recent study on bias in science, of over 1.2 million PhD recipients found that ‘underrepresented groups have to innovate at higher levels to have similar levels of career success’.

Nominating more people from underrepresented groups wouldn’t help when the bar for giving them recognition is much higher than their peers. Isn’t this blatant discrimination? Just having more nominations will become a hollow act and sooner rather than later, people will not be motivated enough. This is an issue with our entire academic world, corporate world and society in general.

Applying Systems Engineering

Just speaking about diversity is a reactive approach to decades of the wrongdoings of bias. While I absolutely believe that we need to be vocal and promote diversity in our teams, organisations and society, the potential impact of all our work could be far-reaching if we are able to create a culture of meritocracy. Recognising people for their contribution (irrespective of their skin colour or race, religion, gender etc.) can be immensely empowering and motivating.

One big thing I have learnt in my career applying systems engineering is that context (by which I mean the environment in which something happens), is important. Actually, context is everything! A command from a controller or a piece of faulty code may be safe or hazardous depending on the context in which they are exposed.

Similarly, circumstances under which individuals’ contributions and achievements were made need to be contextualised. When I say, judge the person on merit, I include their journey towards their achievements (equality is not the same as equity). If we consider just “what” has been achieved, it means we are assuming that everyone in the society has had equal opportunities. And in our current societal setup, I would really like to meet any person who can guarantee that!

There is a need for those at the top of organisations to foster a truly meritocratic environment, and for those within the organisation to celebrate and embrace diversity and create role models. Unless we have this combined approach, there will always remain a glass ceiling for the under-represented and we would have failed to deliver as a society.

Should meritocracy be chosen over diversity?

Truth is I don’t know the answer. However, I believe it’s the trait of a great leader to be able to contextualise the merit (laid on paper) with the fight an individual has had to endure to achieve it, to level up the playing field.

A recent TED post on qualities of an effective team player mentioned hunger as one of the three qualities (the other two being humility and emotional smartness). A few months back, I was speaking with a friend of mine who had to fight throughout her life (due to our societal bias and prejudices), but now that fighting spirit and the drive she has, is her biggest strength. So much so that she wants her son to also undergo some level of hardship to develop that inherent hunger and self-motivation. And I completely agree with her as too much lip service is being made in the name of diversity without creating a meritocratic environment for people to grow or get recognition.

In times to come, our biggest achievement as a society would be if we have created a society where everyone has had equal opportunities and we don’t have to make this decision of meritocracy over diversity or vice versa. Diversity will then be a by-product of meritocracy. However, until then, we need to ensure that we create and celebrate role models who have cut through the hypocrisy of our society and achieved due to their hunger and desire.

An appeal to the research community and industry

When we evaluate applications (be it for awards, research funding, and recruitment), we need to evaluate them in the context of the society we live in and the obstacles the applicants have had to overcome. The academic world is especially poor at this. Unfortunately, the ‘system’ doesn’t really allow for considering what people have experienced. However, the UKRI Future Leaders Fellowships is a scheme which I think does that as it is so focussed on the individuals. And I am not just saying that as a chosen fellow. I still vividly remember a UKRI FLF video (from last year) from Professor Fiona Watt (Executive Chair, Medical Research Council), saying this scheme is for “people who don’t fit neatly into boxes!” Never before, have I heard something like this in an academic world. Kudos to UKRI for creating such a programme.

Before making a decision on applications, just imagine what they could achieve if they were given the same resources and infrastructure (because of their fighting spirit) … If we do this, the number of awardees/lecturers etc. from under-represented sections of the society would increase dramatically, which in turn would inspire many others. However, this is a short term measure.

In the long term, we need to take both a top-down and bottom-up approach to get over the decades-long wrong of ‘biases’. Recognising that the playing field is uneven and that we are not all coming from the same starting point and appreciating this in our decision making would be a major step forward. I hope we can create research and organisational culture which has its foundations on meritocracy enabling diversity.

It is a marathon and we need a few 100m sprints wins to keep our spirits high to drive the long-term commitment to a holistic approach….

................

I must admit, I wasn’t sure how this blog will be received. I was so worried (borderline scared) that I reached out to a few of my friends and colleagues to get their opinion to gauge if I should even publish it. Big thank you to Professor Paul Jennings (WMG), Dr Poonam Goyal (my wife), Corrina Urquhart (WMG), Dr Izzy Jayasinghe (UKRI Future Leader Fellow, University of Sheffield), David Webb (CCAV, HM Government), Michaela Hodges (WMG), Jennie Martin (ITS UK), Nicola Jennings (University of Bath), Luke Logan (BAE Systems), and Dr David Bott (WMG) for their feedback on this blog.

I am an Indian national and a bit outspoken (especially on matters I deeply care), and have been told that if I do the right thing, I would have a good night’s sleep (however long or short it might be!).

This blog represents my personal opinion and not that of my employer or any organisation I am (even remotely) associated with.

**Original article written by Siddartha Khastgir, published to Medium here.