Bodo C. MELNIK
Department of Dermatology, Environmental Medicine and Health Theory, University of Osnabrück, Germany
This review presents recent evidence of impaired Notch signalling in atopic dermatitis (AD), which is proposed to represent the “a-topic” defect linking both epidermal and immunological barrier dysfunctions in AD. AD epidermis exhibits a marked deficiency of Notch receptors. Mouse models with genetically suppressed Notch signalling exhibit dry skin, signs of scratching, skin barrier abnormalities, increased transepidermal water loss and TH2 cell-mediated immunological changes closely resembling human AD. Notch signals are critically involved in the differentiation of regulatory T cells, in the feedback inhibition of activated innate immunity, in late epidermal differentiation associated with filaggrin- and stratum corneum barrier lipid processing. Most importantly, Notch deficiency induces keratinocyte- mediated release of thymic stromal lymphopoietin (TSLP). TSLP promotes TH2 cell-driven immune responses associated with enhanced production of interleukin (IL)-4 and IL-31. Both TSLP and IL-31 stimulate sensory cutaneous neurons involved in the induction of itch. Notably, Notch1 is a repressor of activator protein-1 (AP-1), which is upregulated in AD epidermis. Without Notch-mediated suppression of AP-1 this transcription factor promotes excess expression of TH2 cell-related cytokines. Impaired Notch signalling negatively affects the homeostasis of aquaporin 3 and of the tight junction component claudin-1, thus explains disturbed skin barrier function with increased transepidermal water loss and Staphylococcus aureus colonisation as well as increased cutaneous susceptibility for viral infections. Thus, accumulating evidence links deficient Notch signalling to key pathological features of AD. Key words: AP-1; atopic dermatitis; epidermal barrier; epidermal differentiation; IL-31: innate immunity; Notch; TH2 polarisation; TSLP.
Accepted May 20, 2014; Epub ahead of print May 23, 2014
Acta Derm Venereol
Prof. Bodo C. Melnik, Department of Dermatology, Environmental Medicine and Health Theory, University of Osnabrück, Sedanstrasse 115, DE-49090 Osnabrück, Germany. E-mail: melnik@t-online.de
Emerging evidence supports the view that inflammation in atopic dermatitis (AD) results from inherited and acquired insults to the epidermal barrier. A considerable overlap of disease promoting factors inducing immunological as well as epidermal barrier abnormalities exist in AD (1, 2). This review presents translational evidence for the involvement of deficient Notch signalling resulting in both epidermal barrier and immunological abnormalities in AD.
NOTCH SIGNALLING
Notch signalling plays a key role in the differentiation and maintenance of epidermis, hair follicles and sebaceous glands (3–5). Notch proteins comprise a family of 4 type I transmembrane receptors that influence differentiation processes (6). Canonical Notch signalling is triggered upon the binding of ligands of the Jagged (Jag-1, Jag-2) or Delta (Delta-like-1, -3, and -4) families of a neighbouring signal-sending cell, which leads to proteolytic cleavage of Notch receptors (Notch-1, -2, -3 and -4) of the signal-receiving cell (Fig. 1) (7, 8). Prior to ligand-dependent Notch signalling is a single Notch precursor protein in the Golgi apparatus cleaved through site 1 (S1)-cleavage by a furin-like convertase generating a heterodimeric receptor that is localised in the plasma membrane (Fig. 2) (9–11). Upon ligand interaction Notch undergoes a proteolytic S2-cleavage in its extracellular domain mediated by a disintegrin and metalloprotease (ADAM) (12, 13). The transmembrane-associated Notch is further cleaved by the γ-secretase complex (S3/4-cleavage) releasing its active intracellular Notch (ICN) into the cytoplasm (7, 8, 14). ICN subsequently translocates into the nucleus where it binds to the transcriptional repressor CSL (RBP-J in the mouse) to activate target gene expression (Fig. 2).
Notch signalling is modulated by glycosylation of the extracellular Notch domain (15). The enzyme O-fucosyltransferase 1 (Pofut1) transfers O-fucose to a particular consensus sequence in the epidermal growth factor (EGF)-like repeats of the Notch extracellular domain (16), which is required for efficient ligand binding and signal transduction (Fig. 1) (15, 16). Canonical Notch signalling is important for terminal epidermal differentiation and maintenance of hair follicles and sebaceous glands (17–19).
Fig. 1. Schematic illustration of Notch receptor cleavage. Notch consists of 36 EGF-like repeats. EGF-repeats 11/12 (yellow) are the ligand binding sites. LNR=3 lin12-Notch repeats. ICN (intracellular Notch) consists of an N-terminal RAM (recombination binding protein-J-associated)-domain, an ankyrin domain and a C-terminal PEST sequence. Red arrows indicate Notch cleavage sites: S1=furin, S2=ADAM, S3/4=γ-secretase. The proprotein convertase furin cleaves the Notch precursor protein into a heterodimeric receptor that translocates to the plasma membrane.
Fig. 2. Illustration of canonical Notch signalling. The Notch protein is fucosylated by Pofut1 in the endoplasmic reticulum (ER). Prior to receptor transfer to the cell membrane, Notch is cleaved into a heterodimer by furin (S1) and glycosylated by Fringe in the Golgi. Receptor-ligand interaction between the signal-sending cell and the signal-receiving cell activates S2-cleavage by ADAM metalloproteases (ADAM10, ADAM17) and consecutive S3/4 cleavage by the γ-secretase complex that releases Notch intracellular domain (ICN). ICN translocates into the nucleus and interacts with the transcription factor CSL to activate Notch target genes.
EPIDERMAL NOTCH DEFICIENCY IN ATOPIC DERMATITIS
Remarkably, all Notch receptors are downregulated or even undetectable in the epidermis of lesional skin of AD patients, whereas healthy control patients exhibit significant Notch expression confined to the suprabasal epidermal layers (20) Furthermore, decreased mRNA expression of Notch2, Notch3 and Presenilin1 has been reported in AD skin (21). Notch deficiency appears to be a specific feature of AD as epidermal Notch expression is increased in other inflammatory skin diseases such as psoriasis and lichen planus (20, 22). Loss-of-function mutation of ADAM17 in a sister and brother resulted in eczematous scaly skin prone to S. aureus infection (23).
NOTCH-DEFICIENT ANIMAL MODELS OF ATOPIC DERMATITIS
Ablation of Notch signalling during skin embryogenesis invoked early postnatal death due to a disturbed epidermal barrier and induction of a B-lymphoproliferative disorder associated with enhanced secretion of thymic stromal lymphopoietin (TSLP) (24). Postnatal skin specific inactivation of both Notch1/Notch2 as well as RBP-J in the mouse resulted in an AD-like disease with dry skin, acanthosis, spongiosis, hyperkeratosis, and massive dermal infiltration of eosinophils and mast cells (20). Notably, epidermal Notch deficiency is associated with a significant production of TSLP by keratinocytes (20). Dendritic cells (DCs) activated by TSLP play a major role in TH2 cell polarization associated with the production of the pro-allergic cytokines interleukin-4 (IL-4), IL-5, IL-13, and tumour necrosis factor-α (TNF-α), while downregulating IL-10 and interferon-γ (IFN-γ) (25, 26). Remarkably, Notch-deficient murine epidermis exhibited a 125-fold increased TSLP expression associated with increased numbers of DCs, enhanced expression of IL-4 and IL-13, and a 16-fold increase of serum IgE levels (20). The metalloprotease ADAM10 is preferentially required for Notch1 S2-cleavage (13, 27). Ablation of ADAM10 compromised epidermal integrity, disturbed skin barrier function and upregulated TSLP (28). Keratinocyte-specific ablation of ADAM17 triggered TH2-cell-driven AD (21). ADAM17-deficiency dampened Notch signalling and increased the production of the TH2 cell-polarising cytokine TSLP. Importantly, Notch antagonises c-Fos, an important transcription factor of the activating factor-1 (AP-1) family promoting keratinocyte TSLP expression (28).
NOTCH AND EPIDERMAL DIFFERENTIATION
The strongest genetic association with AD has been shown for loss-of-function mutations in the filaggrin gene (FLG). However, FLG mutations are found in only 30% of AD patients, while around 8% of the healthy population carry identical loss-of-function mutations pointing to another underlying factor involved in AD pathogenesis (29). Notch signalling regulates early-stage epidermal differentiation like the induction of p21 that triggers keratinocyte growth arrest (30–32). Recent evidence points to a pivotal role of Notch signalling in late-stage epidermal differentiation and filaggrin processing (33). Suppression of Notch signalling by RBP-J conditional inactivation resulted in granular parakeratosis, reactive epidermal hyperplasia and increased expression of TSLP (33). Impaired Notch signalling by Pofut1 conditional inactivation led to ultrastructural abnormalities in the granular layer with altered epidermal filaggrin processing. Remarkably, adult Pofut1/Tgfb3-Cre mice displayed full-blown AD-like disease (33). Keratohyalin granules and lamellar bodies (LBs) were significantly reduced and increased levels of monomeric filaggrin have been detected in mutant epidermis (33). Notch deficient murine keratinocytes developed pronounced defects of epidermal barrier integrity and stratum corneum (SC) cornified envelope formation (34). These mice developed an AD-like disease with dry scaly skin, increased transepidermal water loss (TEWL), epidermal hyperplasia with a pronounced inflammatory infiltrate including activated macrophages, mast cells, neutrophils and increased serum levels of IgE (34).
Substantial evidence points to a close relationship between deficient Notch signalling and disturbed filaggrin processing and SC envelope (SCE) formation, important requirements for the attachment of the corneocyte lipid envelope (CLE) that anchors the intercorneocyte barrier lipid lamellae (35).
NOTCH AND BARRIER FUNCTION
Aquaporin 3
Dry skin is a characteristic feature of AD and has been observed in Notch-deficient murine skin (20, 34). Dryness has been related to increased TEWL. Aquaporin 3 (AQP3), the predominant aquaporin in mammalian skin, transports water, glycerol, urea and hydrogen peroxide (36, 37). Increased TEWL in AD has been linked to increased AQP3 expression (38, 39). In healthy skin, AQP3 is mainly expressed in the stratum basale with decreasing levels towards the stratum granulosum (38). Increased AQP3 expression was found in the stratum basale and spinosum of patients with AD (38). Intriguingly, AQP3 has been identified as a transcriptional target of Notch1 (40). Inhibition of Notch signalling increased the expression of mRNA and protein levels of AQP3 (40). Thus, decreased Notch signalling in AD may increase AQP3-mediated TEWL leading to dry skin.
Tight junctions
The junctional complex between epithelial cells is composed of tight-, adherens-, and desmosomal junctions. Tight junctions (TJ) reside below the SC and function as a paracellular barrier (41–43). In AD, reduced expression of the TJ proteins claudin-1 and -23 has been observed (44). Claudin-1 is also expressed in intestinal epithelium where Notch regulates intestinal epithelial homeostasis (45). In a transgenic mouse model, intestinal overexpression of claudin-1 was associated with activated Notch signalling (45). Conversely, low expression of claudin-1 as observed in AD may be associated with deficient Notch signalling. Disruption of TJ barriers with an inhibitory peptide against specific claudins resulted in impaired maturation of lamellae structures, deficient profilaggrin processing as well as decreased non-polar barrier lipids (46). Claudin-1 deficient mice exhibited increased TEWL, altered SC ceramide composition and disturbed filaggrin processing (47). Recent evidence indicates that LB secretion appears to start before the establishment of the TJ barrier (48). Thus, a molecular cross talk exists between Notch and claudins important for the regulalation of epithelial differentiation, TJ formation as well as barrier lipid homeostasis (49).
Barrier lipids
The SC provides highly ordered nonpolar lipid lamellae representing the SC lipid permeability barrier (2, 50). The SC is composed of flattened corneocytes surrounded by multiple stacks of lipid lamellae enriched in ceramides, cholesterol and free fatty acids (50). In comparison to healthy skin, SC of dry skin of AD patients exhibits reduced amounts of ceramides (51). These hydrophobic SC lipids control TEWL. In ADAM10-deleted skin as well as in Notch1 knockouts a twofold reduction of genes regulating epidermal lipid metabolism have been observed (24, 28).
Epidermal lipids are delivered to the SC by LBs (52). The CLE, which anchors the SC lipid lamellae, is covalently attached to glutamine residues of involucrin, the major protein constituent of the CLE (35). ADAM17-deficient murine epidermis exhibited a significant decrease in involucrin and transglutaminase-3 (TGM3) needed for CLE formation and cornification (34). Furthermore, in Notch deficient murine epidermis the number of LBs was significantly reduced (33). Thus, impaired Notch signalling adversely affects filaggrin-, involucrin- and TGM3 processing resulting in deteriorated CLE formation and barrier function. Moreover, Notch signalling activates lipid synthesis by stimulating the kinase AKT and the serine/threonine kinase mTORC1, which enhances the expression of sterol regulatory element binding transcription factor-1c (SREBP-1c), a key transcription factor of lipogenesis (53). Conversely, inhibition of Notch1 reduces the expression of lipogenic transcription factors peroxisomal proliferator-activated receptors (PPAR)-δ and PPAR-γ (54).
NOTCH AND SEBOSTASIS
Decrease of sebaceous gland secretion with reduced sebum-derived lipids such as waxes, squalen and triglycerides also contribute to dry skin in AD (55). Notch signalling is required for sebaceous gland homeostasis (17, 18). Remarkably, Notch-deficient mouse models exhibit absent or reduced numbers of sebocytes with reduced lipid content (18, 28, 33).
NOTCH AND INFLAMMATION
Granulocyte/macrophage colony-stimulating factor (GM-CSF) is a pleiotropic cytokine with multiple effects on dendritic cells (DCs), T cells, monocytes and eosinophils. Prolonged skin expression of GM-CSF induces changes observed in AD (56). Keratinocytes of patients with AD produce high amounts of GM-CSF (57). Increased levels of G-CSF have been reported in a Notch deficient mouse model (21). Conserved lymphokine element 2 (CLE2) and CLE0 are involved in GM-CSF gene transactivation (57). The major transcription factors for GM-CSF gene transactivation are nuclear factor kappa B (NFκB) and AP-1 (57). AP-1 complexes are over-expressed in AD keratinocytes, with higher levels of basal c-Jun and PMA-induced c-Jun, JunB, and phosphorylated forms of c-Fos (57). Most importantly, Notch1 ICN is a repressor of AP-1-mediated transactivation (58). Furthermore, Notch antagonises c-Fos recruitment to the promoters of TSLP and CSF3 (G-CSF) (21). Thus, decreased Notch signalling may upregulate cutaneous inflammation via G-CSF, GM-CSF and TSLP.
TSLP is mainly expressed in epithelial cells and epidermal keratinocytes and is regarded as the most critical cytokine linking responses at interfaces between the body and the environment to TH2 responses including AD (25, 59–61). Mice expressing an inducible TSLP-transgene in the skin spontaneously develop AD exhibiting all cardinal features of human AD (62). Abundant release of epidermal TSLP has been demonstrated in mouse models with compromised Notch signalling mimicking human AD (20, 21, 24, 25, 28). Disturbed Notch signalling may thus induce a TSLP-mediated epidermal “defence response” that maintains a TH2 cell polarised skin inflammation.
Interleukin-31 (IL-31) is mainly expressed by TH2 cells (63). IL-31 is a pruritogen and pro-inflammatory cytokine that is increased in AD skin (64, 65). IL-31 serum levels significantly correlate with disease activity and TH2 cytokine levels in children with AD (66). Recent evidence indicates that IL-31 disturbs epidermal differentiation and appropriate filaggrin expression in human organotypic skin models (67), underlining the potential involvement of IL-31 in the generation of epidermal barrier defects in AD. Raap et al. (66) observed a correlation between serum levels of IL-31, IL-4 as well as IL-13 in AD patients (66). Szegedi et al. (65) reported that IL-31-producing T cells co-produce IL-13. Notably, AP-1 is an important transcription factor required for the production of TH2-related cytokines IL-4, IL-5, and IL-13 (68). In fact, histamine H4 receptor agonist stimulation of TH2 cells induced the production of both AP-1 and IL-31 (69). It is thus conceivable that AP-1 is the critical transcription factor that may enhance the expression of IL-31. Insufficient suppression of AP-1 due to impaired Notch1 signalling may thus be the underlying cause of enhanced IL-31 expression in AD skin. Furthermore, insufficient suppression of AP-1 in Notch-deficient AD skin promotes keratinocyte-mediated TSLP expression explaining TH2 polarization associated with enhanced expression of TH2-related cytokines including IL-31.
NOTCH AND INNATE IMMUNITY
Macrophages that express pro-inflammatory cytokines accumulate in inflamed skin of AD (70). House dust mite allergen Der p 2 and nickel activate macrophage Toll-like receptor (TLR)-4 (71, 72). S. aureus membrane fragments induce TSLP in human keratinocytes through the TLR2/TLR6 pathway (73, 74). Friction and occlusion typically enhanced in flexural skin folds increase the release of IL-1β and tumour necrosis factor (TNF)-α (75, 76). Both inflammatory cytokines are up-regulated in the skin of AD and Notch-deficient murine epidermis (20, 77, 78). IL-1β amplifies TLR signalling because the IL-1 receptor (IL1R) and TLRs share the common intracellular Toll/IL1R (TIR) signalling domain required for downstream activation of mitogen activated protein kinases (MAPKs) (79). MAPKs are negatively controlled by MAPK phosphatase-1 (MKP-1) (80). Remarkably, Notch signalling upregulates MKP-1 expression (81–83). Thus, Notch inhibits TLR-triggered inflammatory macrophage responses, revealing a new mechanism for negative regulation of TLR signalling by the Notch pathway (84). Thus, deficient Notch signalling may result in a persistent pro-inflammatory activation state of macrophages and DCs as observed in AD (70).
NOTCH AND REGULATORY T CELLS
Thymus-derived and inducible CD4+CD25+FoxP3+ regulatory T cells (Tregs) play an important role in the regulation of allergic diseases (85–87). Children with lower Treg numbers at birth have a higher risk to develop AD and sensitisation to food allergens during the first year of life (88). FoxP3+ Tregs in peripheral blood of children with AD and/or food allergy were significantly lower than in patients without these symptoms and inversely correlated with serum IgE levels (89).
Surface molecules of thymic epithelial cells (TECs) are essential for thymic T-cell differentiation (90). Among these, Notch1 and Notch3 are most critically involved in thymocyte maturation and thymic Treg differentiation (91, 92). Remarkably, ADAM10-deficient mice exhibited huge vacuoles in TECs and reduced thymocyte numbers (28). Notch1 and transforming growth factor (TGF)-β1 cooperatively regulate FoxP3 expression important for Treg differentiation and function (93). Activation of Notch1 has been found to be a novel mechanism in the induction of human Tregs mediated by mesenchymal stem cells (94). FoxP3 is the most critical regulator for the development and function of Tregs (95, 96). Notably, FOXP3 has been identified as a downstream target of Notch (94). Additionally, Notch enhances the expression of IL-2 receptor (CD25) (97), required for IL-2-mediated activation of signal transducer and activator of transcription (STAT)-5 on the FOXP3 promoter (98). Notch signalling is thus critically involved in Treg differentiation and function.
NOTCH AND INFECTIONS
AD patients exhibit an increased susceptibility for viral infections like herpes simplex, mollusca contagiosa and common warts. Interferon (IFN)-γ, the signature cytokine of TH1 responses, plays a central role in antiviral defence. Skin-infiltrating T cells in AD exhibit significantly less IFN-γ production compared to controls (99). Noteworthy, the IFN-γ gene (IFNG) has recently been identified as a direct target of Notch1 (100). Compromised Notch signalling may thus explain the increased susceptibility for viral skin infections in AD.
S. aureus has a peculiar ability to colonise the skin of AD patients (101). Notch signalling plays a crucial role for appropriate filaggrin processing (30–33). Notably, filaggrin breakdown products contribute to low pH at the skin surface (102), and reduce S. aureus growth by decreasing the expression of bacterial factors involved in colonization such as clumping factor B and fibronectin binding factor B (103, 104). Notch-dependent disturbances of epidermal barrier integrity and filaggrin processing may thus predispose AD skin to S. aureus colonisation. In fact, ADAM17 loss-of-function mutation in 2 humans was associated with cutaneous S. aureus infection (23). Staphylococcal enterotoxin B (SEB) amplified IL-31 mRNA expression in polarised TH2 cells (69), which has been associated with compromised filaggrin processing (67). Moreover, peripheral blood mononuclear cells of AD patients upon stimulation of staphylococcal α-toxin and SEB secreted significantly more IL-31 (105).
NOTCH AND PRURITUS
Pruritus is the cardinal symptom of AD. Itching reflects a distinct quality of cutaneous nociception elicited by stimulation of neuronal receptors at the superficial layers of the skin and mucocutaneous orificies. Notch deficiency in keratinocytes of mice results in substantial release of TSLP associated with signs of scratching (20, 21, 24, 28, 33). Intriguingly, keratinocyte-derived TSLP has recently been shown to stimulate cutaneous sensory neurons to promote itch (106). Thus, there may be a direct link between epidermal Notch deficiency and TSLP-induced pruritus in AD. AP-1 not only stimulates TSLP-expression but apparently also IL-31. The recently identified IL-31RA, which is expressed by a small subpopulation of IL-31RA(+)/TRPV1(+)/TRPA1(+) sensory neurons, links IL-31 to the generation of TH2 cell-mediated itch (107). As Notch negatively controls AP-1 activity both sensory pathways of itch transmission might be affected in Notch deficient AD skin.
CONCLUSION
Accumulating evidence points to disturbed epidermal differentiation with impaired skin barrier function as the primary cause of AD (108). The Notch cascade orchestrates epidermal differentiation and barrier function (Fig. 3) (8). Notch deficiency in AD epidermis and mouse models both up-regulate the alarmin TSLP, which induces TH2-driven responses and TSLP- and IL-31-mediated itching (59–62, 105). TSLP activates an intrinsic (atopic) epithelial defence cascade, which is inadequately down-regulated due to deficient Notch signalling. Keratinocyte-derived TSLP and GM-CSF activate macrophages and DCs driving TH2 polarisation promoting the recruitment of eosinophils and mast cells (56, 57, 59–62). TH2-polarised cells induce the pro-inflammatory IL-31 that is closely associated with AD immunopathology (63–69, 107). Insufficient feedback inhibition of innate immunity is another pro-inflammatory aspect of impaired Notch signalling (84). Of crucial importance are alterations of late epidermal differentiation and filaggrin processing (33). Decreased Notch signalling negatively affects key elements of skin barrier function: AQP3-controlled TEWL, TJ-regulated paracellular barrier function and SC lipid barrier function (24, 28, 33, 40, 45). The defective barrier in AD may enhance allergen penetration, sensitisation and S. aureus colonisation further triggering innate immunity. S. aureus exotoxins finally enhance IL-31-mediated proinflammatory signalling and itch transmission (69, 106, 107). Increased viral susceptibility may be explained by decreased IFN-γ expression and insufficient Treg-mediated suppression of TH2 polarisation due to Notch deficiency (87, 89, 93). Notch deficiency via upregulation of TSLP and IL-31 appears to generate pruritus, the clinical hallmark of AD (106, 107). Taken together, deficient epidermal Notch signalling is closely related to key features of AD pathogenesis (Table SI1).
Future studies should focus on the molecular crosstalk between Notch and epidermal inflammation, differentiation, barrier function, antimicrobial responses, and treatment regimens in AD patients.
Fig. 3. Synopsis of deficient Notch signalling (DNS) in the pathogenesis of atopic dermatitis. Insufficient Notch-mediated expression of IFN-γ explains increased susceptibility for viral infections. Impaired-ICN-mediated repression of AP-1 explains up-regulation of GM-CSF promoting skin infiltration by eosinophils (Eos), dendritic cells (DCs), epidermal Langerhans’ cells (LCs) and mast cells. Impaired Notch-mediated suppression of keratinocyte-derived AP-1 explains increased TSLP expression with TH2 cell polarisation with enhanced TH2 cell-mediated IL-4- and IL-31 expression and enhanced TSLP- and IL-31-mediated neuronal transmission of itch. DNS disturbs filaggrin processing, which affects skin surface pH and suppresses clumping factor B (CLF-B) and fibronectin binding factor B (FBF-B) resulting in enhanced S. aureus colonisation. DNS reduces MAPK phosphatase-1 (MKP-1) expression, resulting in insufficient feed back inhibition of innate immunity. DNS may decrease claudin-1 and tight junction (TJ)-controlled paracellular barrier function and activates AQP3-expression increasing TEWL. DNS is associated with reduced numbers of LBs and suppression of key enzymes of epidermal lipid metabolism.
DEDICATION
This review is dedicated to my academic teacher of dermatology Prof Gerd Plewig, University of Munich, on the occasion of his 75th birthday.
The author declares no conflict of interest.
1http://www.medicaljournals.se/acta/content/?doi=10.2340/00015555-1898
REFERENCES