Editor’s note: This text-based course is an edited transcript of the webinar, Understanding and Treating Collective Trauma in the Age of Covid-19, presented by Tiffany Hall, MS, LMFT.
After this course, participants will be able to:
- Define and distinguish collective trauma from individual trauma
- Appraise the biopsychosocial factors impacted by the Covid-19 pandemic and different types of trauma responses
- Consider potential clinical interventions to help address mental health challenges caused by Covid-19 pandemic
This presentation was created at the end of August 2020. Please note that this presentation will discuss the current trauma as evidenced by COVID-19 at this moment in time. The true realities of the impact of COVID-19 on trauma and mental health are unknown at this time because the situation remains ongoing. I hope that this course will give you some insight into how to help clients who are struggling with some of the unique challenges posed by the COVID-19 pandemic.
What is collective trauma?
Collective trauma refers to a traumatic event that is experienced by a group, community, or society. A traumatic event can often cause long-term psychological symptoms within the group. In many cases, these psychological effects have a hand in shaping the community long after the traumatic event occurs. People within the group will experience their own individual trauma as well. These individual trauma symptoms can also contribute to the experience of collective trauma.
Types of Collective Trauma
There are different types of collective trauma events. These are the most common examples that we see collective traumatic events:
- Terror attacks
- Natural disasters (i.e. earthquakes, hurricanes, etc.)
- Mass shootings/violence
- Civil unrest
Events like these can create a feeling of instability and unsafety within the community or a group. Every society has experienced one of these events at some point in their history.
Intergenerational and Historical Trauma
Intergenerational and historical trauma are another type of collective trauma. It is experienced by subsequent generations after the traumatic event has occurred. This can be seen in Holocaust survivors, children of immigrants or refugees, and survivors of mass violence.
What causes subsequent generations to experience this trauma if they were not around to witness it firsthand? It is usually a secondary trauma response. This includes images or retellings of the traumatic experience from members who did experience it. Those that survived the trauma may be living with complex post-traumatic stress disorder (PTSD) and that affects the way that they interact with members of their community and their families.
Traumatic responses may evolve into cultural norms. For example, the September 11th terrorist attacks are a traumatic event that has a significant impact on the generations that have followed it. September 11th attacks initially resulted in a sense of community and enhanced national pride in the United States. It also led to widespread discrimination of people of Middle Eastern descent and Muslims, who were depicted as terrorists.
The Homeland Security Act was passed due to the prioritization of national security and safety. Two years following the attacks, the Department of Homeland Security was formed. The attacks and the result in collective trauma ultimately reshaped the airline industry.
The 1918 Flu Pandemic
The most recent pandemic that we have experienced similar to the one that we are experiencing right now is the 1918 Spanish Flu. Today, the pandemic has caused mass devastation and loss. Schools and businesses have closed, people are resistant to wearing masks, and the economy has suffered.
Medical interventions such as vaccines and antivirals were not available as they are today to treat this infection. Secondly, public health and hygiene practices at the time made it easier for viruses to spread. The pandemic also occurred concurrently with World War I, which aided in the virus’ global spread. It was also another collective traumatic event, adding to the experience of trauma from the pandemic.
Similar to present times, people back then experienced collective trauma caused by mass illness, death, disruptions to life, and threats of a virus many did not understand. There is little research about the long-term mental health impact of the 1918 pandemic. However, some research has shown a correlation between the influenza death rate and suicide in the United States between 1918 and 1920.
Collective Trauma Responses
Collective trauma responses can be categorized into four different types, shown in Figure 1:
Physiological responses refer to the way that the body responds to trauma. Examples of physiological trauma responses include rapid breathing, increased heart rate, headaches, digestive issues, and muscle tension. Physiological trauma responses are often immediate and are usually the first indicator that one is reacting to a threat. Those who experienced physical symptoms may not be aware that what they are experiencing is connected to their psychological experience of the trauma.
Cognitive trauma responses refer to the ways in which one's thoughts and beliefs are affected by the trauma. Collective trauma can influence the way in which we view the world, as well as ourselves. It can also impact the ways in which the brain processes information. Nightmares, flashbacks, and negative self-talk are examples of cognitive responses to trauma.
Emotional responses are the ways in which feelings and emotions come from reacting to trauma. Collective trauma can cause a wide range of emotional responses. Many people experience sadness, fear, anxiety, frustration, or other emotions caused by exposure to trauma. The feelings can be overwhelming for those experiencing them and may cause disruption to their lives.
The cognitive and emotional responses to trauma can ultimately influence behavioral responses. Behavioral responses to trauma are often ways that we adapt to the new sense of normal caused by the trauma. One of the most common behavioral responses to trauma is hypervigilance, which is when those who have experienced trauma are sensitized to detecting a threat. They may have a hard time relaxing and can feel paranoid or unsettled. Avoiding things that remind one of the trauma is also a common behavioral response.
Contributing factors to trauma responses
Figure 2 shows the different ways trauma responses can be impacted:
Due to the fact that each individual responds to trauma differently, many people who are exposed to trauma may not end up experiencing PTSD symptoms. The way that one responds to trauma is largely influenced by multiple factors.
Adverse childhood experiences such as exposure to violence, poverty, abuse, or neglect can have a significant impact on one's socio-emotional development. Many who have experienced adverse childhood experiences have higher rates of chronic, mental, and physical illness. Additionally, a lack of resources and support from the community also exacerbates trauma responses. People who experienced childhood trauma may be more primed for hypervigilance, negative self-perception, and an overall feeling of unsafety in the world. All of these combined can make a community much more vulnerable to collective trauma.
PTSD and Complex Trauma
Figure 3 represents the factors within PTSD as well as complex PTSD:
Adverse childhood experiences can also contribute to the development of complex PTSD. Complex PTSD is slightly different from PTSD in that it is not the response to just one specific event. Instead, complex PTSD is a response to a series of traumatic events that have occurred throughout a lifespan. For instance, a person who has experienced both poverty and family instability have already experienced anxiety and hypervigilance. Experiencing additional traumatic events may exacerbate preexisting traumatic symptoms. These symptoms may be compounded and more difficult to manage over time.
How does COVID-19 make an impact? The pandemic is an event that is ongoing. Thus, we are still learning about the long-term effects. What we do know is that COVID-19 is unlike many other traumatic events that we have experienced in our lifetime. There have been several other pandemic events in modern history, specifically in the 20th and 21st centuries. However, this is the first pandemic that many people have experienced at once. Also, it is one of the first to cause such broad disruptions to daily life for a long period of time.
The ongoing nature of COVID-19 makes it more challenging for people to cope. One reason for this is that there seems to be no foreseeable end. This is different from a terrorist attack or natural disaster, which are one-time events that are widely visible. Additionally, COVID-19 is often referred to as an invisible enemy. Unlike a war or other visible threat, COVID-19 is caused by an invisible pathogen that we cannot see without the aid of a microscope. I bring this up because, cognitively, many people struggle to process a sense of danger caused by this disease. Widespread lags, a lack of access to diagnostic testing, mixed messages from government and health officials, and misinformation have contributed to many people having difficulty gauging the severity.
Furthermore, many countries have imposed lockdowns and social distancing guidelines that force people to spend time at home, away from loved ones and community. Shutdowns of businesses and schools also mean that many people are not receiving the same amount of social stimulation that they are accustomed to.
In addition, the ongoing nature of the pandemic means that there is no perceived end. We do not yet know how long the pandemic will last or what the outcome will be. The loss of that sense of normalcy, as well as the lack of predictability, can enhance feelings of fear and anxiety.
Also, mass illness and death, on top of economic instability, create a sense of widespread loss and suffering. This sense of loss creates collective grief. Nearly everyone has lost something due to COVID-19, whether it is a loved one, a job, routine, or a sense of normalcy.
Primary vs. Secondary Trauma Responses and COVID-19
Though the pandemic has affected everyone in some form, the experience of trauma is also based on one's proximity to it. For instance, those who are healthcare or frontline workers may be directly witnessing the loss and devastation caused by COVID-19 on a daily basis. Many of these workers are already exhibiting symptoms of PTSD or other mental health challenges. These may be primary trauma responses, which can be seen in individuals and groups. They are normally the immediate or short-term responses to a traumatic event.
Those who experience the trauma indirectly or through secondary exposure are likely to develop secondary trauma symptoms. This can mean experiencing secondary trauma from consuming media about the pandemic or witnessing the impact on their communities. Caregivers are especially prone to developing secondary trauma as they provide care to those who were traumatized.
Figure 4 shows how the collective experience of trauma can happen:
There are extreme external stimuli or the actual traumatic event itself. We experience this through our own traumatic responses. Emotional, physiological, cognitive, and behavioral are depicted here as neuro-psychological, physiological, emotional and sensory. Some of the trauma responses may later develop into PTSD.
Also, individuals and groups had their own set of coping skills to manage the discomfort that can result from the trauma responses. Coping skills may vary based on the resources and supports that are available. Coping skills are further supplemented by meaningful relationships, community, and finding a sense of meaning from the traumatic experience. Be mindful that this process could take weeks, months or even years, and is not necessarily a linear process.
Potential Social Impacts of Pandemic
Although we are still learning about the lasting impacts of the pandemic, it is probable that the mental health challenges will exist long after the pandemic has ended. Some experts have also described the psychological impact as being a secondary pandemic. Additionally, a tense and sociopolitical climate in the U.S. has contributed to divisions in national identity. Many of these divisions existed prior to the pandemic and had been further exacerbated. In times of national crisis, a sense of unity can provide comfort and belonging, which is something that many people feel is missing right now.
Due to the spread of misinformation about the virus, there is a lack of trust in our public institutions. Normally, public health institutions like the Centers for Disease Control (CDC) and the World Health Organization (WHO) lead the charge in providing information to the public about the disease and how to combat it. However, the COVID-19 pandemic has become politicized. This has led to public health professionals’ roles being diminished. As a result, there has been contradictory messaging on how to manage the disease or how severe it is. Government and health essentials need to work together to communicate in order to provide a sense of safety.
We may also see a shift in the ways we work. As many businesses move to a telework model, many people now have the opportunity to work from home. Both employers and employees can benefit from working remotely. Many employers are also able to reduce operating costs that come with having an office space. Additionally, employees now have reduction in commute time and costs. Some report an increase in focus, productivity, and work-related stress. While many businesses will not be able to sustain a long-term telework model, others may continue to incorporate it into their employment structure.
While the ability to work remotely has been beneficial for many, it has created more challenges in other ways. Due to closures of schools and childcare facilities, parents and caregivers are often forced to manage the tasks of work and household duties in childcare. This can feel like an impossible balance for many families and can enhance stress.
One of the most apparent effects of the pandemic is fear. COVID-19 is still a novel disease, which means there is still a lot that we do not know. We have learned that the virus can cause serious illness. We have also learned that a large portion of those who have been infected show no symptoms at all. Additionally, we have learned that the elderly and immunocompromised populations are especially vulnerable to this disease. However, many people who had no pre-existing health issues have also become severely ill and died.
We are currently seeing the scientific method happen in real time. The frequent changes in expectations, feelings of uncertainty, and the lack of trust in public institutions can be distressing and confusing. Thus, prioritizing mental health care is important during this time.
When working with clients experiencing stress or trauma, clinicians should work to normalize their feelings. Collective trauma can elicit a wide range of emotional responses that may be difficult for many people to understand. Clinicians should affirm clients and allow them a space to process their feelings.
Many clients will need assistance in identifying their feelings. Since the experience of trauma can be overwhelming, it may be challenging to describe and explain any psychological experience. Clinicians should help clients identify the ways in which they can cope with their feelings. Help clients realize who their support system is, what gives them joy, and what helps them to feel calm or safe. Doing this may also help clients identify the areas of their lives in which they have control. This is important because collective trauma can make one feel out of control, especially if we do not know how long we will have to endure a situation. The virus itself defies all of our expectations and we cannot control it.
Clinicians should also encourage clients to find community. The pandemic has limited our ability to gather in groups. Therefore, many people do not have access to their usual social outlets. However, there are still plenty of ways to stay connected with others while adhering to social distancing guidelines. Thanks to technology, we have the ability to connect with people all over the world. Clients can access virtual support groups and classes. Many therapy practices also provide telehealth for the opportunity to safely access mental health treatment. Social media can provide a way for people to share their experiences and feel connected to a community as well.
Working with Vulnerable Populations
People who are especially vulnerable due to past trauma will experience collective trauma much differently.
Treating Children and Adolescents
Children and adolescents are especially vulnerable to the effects of collective trauma. Due to the increased stress on families, children are at a much greater risk for being victims of abuse or neglect. School and childcare closures mean that many incidences are going unreported. It is important that providers fully assess any signs or risks that may show abuse is occurring. Depending on their developmental age, children may be limited in their understanding about what is happening during this pandemic.
Children may also be struggling with the changes to their routine and isolation from peers. The impact of collective trauma on children will be influenced by their caregivers, environment, and resources. Children often take their cues from their caregivers and have the ability to be resilient. Clinicians can work with caregivers to find appropriate ways to help children cope with uncomfortable feelings.
Thus, it is important that parents or other caregivers be mindful of how they model coping behaviors. If parents are stressed, children may be able to sense this. This can then impact their own coping abilities. Parents and caregivers should also be assisted in identifying other resources for support. Are there other family members or loved ones that can help with childcare? Are there other programs that provide resources?
Other Vulnerable Populations
Trauma tends to have a more severe impact on other marginalized communities:
- Black and brown communities
- Poor and under-resourced communities
- People living with disabilities
- Older adults
Black and brown communities, which are disproportionately poorer than white communities, are also likely to be more impacted by the pandemic. We already know that black and brown communities have a higher likelihood of contracting the virus and being hospitalized for it. These communities are also dealing with racial trauma, over-policing, violence, and a lack of resources.
People also may be more exposed to COVID-19 due to many frontline workers being in jobs that do not allow them the option to work from home. Many may also live in smaller and tighter living spaces with multiple household members. The choice between making an income and lowering the risk of contracting the virus by staying home is a difficult one. For many, it is simply not a choice at all. This is because if they do not work, then they will not be able to provide for themselves and their families. Marginalized groups also experience disproportionate rates of homelessness, unemployment and underemployment, and domestic violence. Children and other marginalized persons may not be a safe staying at home, as many reside with people who harm them.
Marginalized populations may also have limited options for mental health treatment. The availability of telehealth broadened the reach of mental health care. However, it is not accessible to everyone. Some may not have Internet or computer access. Also, some people may not be able to use the various devices required to engage in telehealth services due to disability or other difficulties.
While treatment with a licensed mental health professional is recommended, psychotherapy is one of many tools that one can use to care for their mental health. Additionally, spiritual and religious practices are a common way that many people find community and coping mechanisms. Physical activity and creative outlets also may provide a way for people to express their feelings and reduce tension.
Everyone experiences trauma differently and there are simply not enough equitable resources available in helping marginalized communities. Providers should work with clients to identify the appropriate resources available to them, even if it does not include traditional forms of mental health treatment.
Social Media Use and Mental Health
Social media can be a great coping tool for people who are dealing with trauma, but can also exacerbate those experiences. Social media allows us to have access to people outside of our immediate communities and connect with them from all over the world. Thus, it can help manage feelings of isolation caused by social distancing and sheltering in place. It may help one to normalize their trauma experience because social media gives many people a space to share what is happening in their lives. This can also create feelings of belonging.
However, social media can worsen mental health challenges as well. Social media allows us to quickly access information, which often means there is a risk of seeing traumatic material. In an attempt to gain a sense of control and stay informed, many people over-consume media related to the COVID-19 pandemic. They may engage in political discussions, which can further exacerbate feelings of frustration, sadness, or distress.
It is important to be mindful of how much and what kind of media one consumes. This is because it can impact the way that they experience trauma. Providers may suggest that clients reduce the amount of time that they spend consuming media. They may also advise clients to be mindful of the people they follow or how much they engage in discussions around certain topics.
Substance Use and COVID-19
The increased stress from the pandemic is causing many people to look for relief. Therefore, substance abuse has increased, which can pose other challenges.
First of all, opioid shortages in some places may cause users to find alternative substances or find another method. People in recovery may find it more challenging to maintain their sobriety, as the stress of the pandemic may cause their coping skills to be less effective. Those in active addiction may increase their frequency or amount that they use. Providers should screen clients for excessive substance use or alcohol use. Substance use may also lead to health complications that make a person more vulnerable to contracting COVID-19.
COVID-19 and Medical Trauma
In the case of this disease, the reality is that we or some of our loved ones will contract the disease. Receiving a COVID-19 diagnosis can potentially lead to serious illness or death, and people can experience anxiety about their health status. In more severe instances, some people may become preoccupied with cleanliness or disease control to the point that they develop obsessive-compulsive coping behaviors. It is important that providers screen for that.
Experiencing the trauma of mass illness and death may contribute to examining our own feelings around dying. What happens if I get sick? How will I pay for the medical cost? What if I die? Who will take care of me or my family? These may be some of the questions that people ask themselves when they think about the possibility of getting sick with this virus.
The death rate currently remains low and most who become ill will likely survive. However, some may still deal with the lasting trauma from serious illness and hospitalization. Some can experience depression, anxiety, or develop PTSD symptoms. Those who have been hospitalized may experience something called post-intensive care syndrome. This can consist of emotional, cognitive and physiological symptoms that may cause impairment to daily life.
Normalizing Grief and Loss in Response to COVID-19
It is important to normalize the experience of grief. All of us have lost something to this COVID-19 pandemic. Grief is a natural response to loss and is necessary for trauma recovery.
Similar to trauma, grief can be an overwhelming experience. If we look at Elisabeth Kübler-Ross' five stages of grief, we see that there are different emotional stages that one experiences during the grieving process:
We can apply these stages to the pandemic by identifying and normalizing the emotions that we experience.
The first stage is denial, which is often a way to manage the shock and emotional pain that comes with loss. People stuck in a state of denial may minimize the seriousness, insist on ignoring social distancing guidelines, or refuse to wear a mask. Many people in the denial stage may not be ready to accept the reality that life prior to the pandemic is never going to be the same.
Another stage is anger. This is when people may express feelings of resentment at government institutions, social distancing guidelines, or at the virus itself. Anger can show up as feeling that the world is unjust and the pandemic is causing suffering to those who do not deserve it. Some may feel angry that the virus has taken away a loved one, a business, or plans that they looked forward to.
In the bargaining stage, one may try to rationalize the loss to try to make sense of it. People may wonder how they could have prevented it from happening. They may think that if they had done something differently, perhaps it would not have happened. It can be hard to make sense of something like a pandemic.
The sadness stage occurs when the gravity of the loss begins to sink in. One may be sad because they miss their loved ones, their workplace, or a normal routine. They may be sad that so many people have died. They may also feel a sense of powerlessness to do anything about the suffering happening around them. Sadness that persists for a prolonged period of time can eventually develop into clinical depression.
Finally, there is the acceptance stage. This is the stage that many of us are trying to get to, which is accepting that the loss has happened and what was lost cannot return. A new normal is then created from the loss.
It is important to note that the grief process is not linear. It is possible to experience all five stages within one day or week. For many people, this process can take years.
Considerations for Providers
Figure 5 represents some considerations that providers should keep in mind:
Providers should be mindful of the way that collective trauma is impacting us as well. Experiencing collective trauma means that we are having to manage our own trauma responses while providing care to others. Providing care via telehealth may also pose challenges because providers need to balance work and family responsibilities. There are also technical issues and social isolation from colleagues.
Providers should also be aware of the signs of compassion fatigue and burnout. Providing care in the midst of a crisis is not the same as providing care in more normal times. Burnout symptoms may be similar to trauma responses. Thus, providers may have less stamina to manage their workload and stay engaged with their clients. This can be challenging, as many of us are considered essential workers and may not have the opportunity to take time off.
It is important that providers prioritize self-care when we can. Scheduling time off, participating in our own therapy, or doing activities that we enjoy are great ways to manage stress. Many of us have received no training in our education around how to provide care in a pandemic. This means that we are adapting with our clients.
Providers should also find ways to connect with their supports. Making time to talk with loved ones or colleagues can help manage feelings of isolation and normalize our own experiences.
Providers should also be aware of their professional limitations. Dealing with our own trauma means that we may not always feel effective in treating others. We are also relying on the resources available to us, and yet there is still a lot that we do not know. This is an experience that we share with the people that we serve. We are all learning and adapting each day to this pandemic. As we continue to learn about COVID-19, we will learn more about the challenges that it poses. A shared experience of the trauma can help us to feel less alone, which can help us to recover.
Remember that human beings are adaptable and resilient. Our species has lived through periods of loss and devastation throughout history and we have managed to evolve and survive. While things may seem uncertain now, we can rest-assure that it will not be this way forever. However, we need to be prepared for the long-term psychological impact that COVID-19 may leave.
Questions and Answers
What have you noticed in your clients since the pandemic?
Anxiety seems to be the most common thing that I am seeing in my practice. It can cause us to feel a lack of control and a lot of people are experiencing that right now. We still do not understand a lot about this virus or how long we are going to be living under these conditions. There are now people who are struggling to adjust to life at home. Plenty of people I have spoken with are having privacy issues and need to find somewhere else to do their therapy sessions. There are now certain industries who are going to have a difficult time recovering from this. I have plenty of clients who have lost jobs or reduced income due to.
You brought up your clients not having ample privacy for therapy. What is the workaround for that?
Technology makes it possible for us to access telehealth anywhere. Clients are able to use their phones or other devices that allow them to be mobile, which has been helpful. Some people may decide to go to a friend's house or their car to do sessions. However, going back to anxiety being a common diagnosis, remember that all of us are also dealing with everyday stressors that come with just being alive. On top of those stressors there is also the ongoing pandemic, so we are being squeezed from all sides.
I have often thought about clients who are children, college students, or some adults in which home may be a source of trauma. What do you recommend that we do as practitioners when we are working with these clients?
One of the most challenging aspects of the pandemic is that not everyone has the same level of privilege and resources. While we are protecting ourselves from the virus, being at home for many people causes a threat to their sense of safety. They may be living with people who are abusive or neglectful in some way. This is especially difficult for children who do not have the ability to remove themselves from their living environment. Providers should screen to make sure that the abuse and neglect is not persisting. There may be grandparents or other providers that they can develop relationships with as well. For a lot of our clients, social workers are some of the most consistent people that they have at the moment.
Being in therapy or having a routine is a great way to feel a sense of safety. However, it is not an easy thing to do. It is dependent on the amount of resources that they have available to them. Most of the college students I work with have struggled with abruptly leaving campus. Many of them have not lived at home for a couple of years and grew accustomed to living independently. While many of us have loved ones that we live and feel safe with, it is not good to spend time with the same person all day and every day. However, it is a privilege to be at home with people who make you feel safe because not everybody has that. This is especially unfortunate for children. That is why we have to be mindful of the ways that parents and caregivers are coping. They are modeling behaviors for children. If parents and caregivers are stressed out or are having a hard time coping, the children may pick up on that.
Can you break down PTSD and complex trauma a bit further?
Complex PTSD is more severe than regular PTSD. PTSD usually occurs in response to a traumatic event. For example, most people associate PTSD with veterans returning home from combat. Complex PTSD is more chronic because it is a response to a series of traumatic events. It can be more severe for people who have had adverse childhood experiences. The person that caused the trauma may be a caregiver themselves. Complex PTSD usually develops overtime and can affect one's aspect regulation. People with complex PTSD often struggle with low self-esteem, trusting others, managing their own feelings, and mood swings. It is a more compounded and severe version of PTSD, but it can be treated.
What would your recommendations be for treating PTSD or complex trauma?
Complex PTSD can be treated with cognitive behavioral therapy. This can help someone process their trauma responses. Eye movement desensitization and reprocessing (EMDR) is another way to treat trauma. It is designed to reduce the intensity of the trauma responses. Some people feel that EMDR is effective in a shorter period of time rather than regular talk therapy, but it is not necessarily a good fit for everyone.
Also, get involved in activities that allow for emotional or creative expression. Journaling is a great way to process feelings and track one's healing process. Another thing to do is find community. It is important to connect with others because we are currently isolated and stressed about what is happening. We want to have people who we can vent to or who can affirm our experiences. This can help reduce the intensity of these traumatic responses.
Do you have any case examples that could help us in working with clients who are impacted by collective trauma in the midst of COVID-19?
I have a client who is a high school student and has been struggling with isolation from peers, having to be at home, and not having structure. I have been working with them on how to be social in a safe way. They have been able to identify one or two peers that they can spend time with, either face-to-face or remotely.
I have also encouraged them to create their own routine. Having a routine is helpful because the line between our personal lives and our work lives is now gone. Creating a routine is a way to break up the two. Having a defined start and end time for work or school can help break up the time and give some normalcy to what is happening. Even though this client is not physically in school right now, they are making sure that they sleep at the same time every day and have a friend who they see every week. This gives them a sense of routine to help calm down some of the feelings of displacement and instability.
Refer to the course handout for a complete list of references.