<?xml version="1.0" standalone="yes"?>
<Paper uid="W04-0103">
  <Title>A Diachronic Approach for Schwa Deletion in Indo Aryan Languages</Title>
  <Section position="3" start_page="0" end_page="1" type="metho">
    <SectionTitle>
2 The Problem
</SectionTitle>
    <Paragraph position="0"> Schwa is defined as the mid-central vowel that occurs in unstressed syllables. The first vowel of the IAL alphabet {a}  is the schwa. Normally, it is pronounced as /@ / in Hindi and Sanskrit, and as /o / in Bengali. Schwa deletion is a phonological phenomenon where schwa is absent in the pronunciation of a particular word, although ideally it should have been pronounced (Ohala, 1983).</Paragraph>
    <Paragraph position="1"> Sanskrit and some of the modern IAL that have evolved from it (e.g. Hindi and Bengali), are written from left to right using Brahmi-derived scripts. All the vowels are explicitly represented using diacritical or non-diacritical marks around the consonant except for the schwa, which is the inherent vowel. Unlike Sanskrit, many modern IAL like Hindi and Bengali allow deletion of schwa in certain contexts. Table I illustrates this phenomenon for the three languages. In order to determine the proper pronunciation of the words, it is necessary to predict which schwas are deleted and which are not. Thus, schwa deletion is an  The graphemes for Indo-Aryan languages are written within '{' and '}' according to the scheme adopted by the International Congress of Orientalists at Athens in 1992. The phonetic transcriptions are written within two '/' using the  in three different IAL. The number of syllables is denoted within parenthesis below the pronunciations. In Bengali {a} can also be pronounced as /o/ in certain contexts.</Paragraph>
    <Paragraph position="2"> Several theories have been proposed on the linguistic aspects of schwa deletion in Hindi (Pray, 1970; Kaira 1976; Ohala, 1977, 1983) and its diachronic evolution (Misra, 1967). Ohala (1983) has summarized the rule for schwa deletion in Hindi as @ - ph / VC __ CV Condition 1: There may be no morpheme boundary in the environment to the left.</Paragraph>
    <Paragraph position="3"> Condition 2: The output of the rule should not violate the phonotactic constraints of Hindi Convention: The rule applies from right to left The explanation of the rule was based on psycholinguistic evidence; diachronic facts were used only to explain the exceptions. Narsimhan et al (2001) designed an algorithm for schwa deletion in Hindi based on this work. The reported accuracy of the algorithm is 89%. Some rules for word final schwa deletion in Bengali have been proposed by Chatterji (1926), but we do not know of any work on computational modelling.</Paragraph>
  </Section>
  <Section position="4" start_page="1" end_page="1" type="metho">
    <SectionTitle>
3 Factors governing language change
</SectionTitle>
    <Paragraph position="0"> The fact that schwa deletion in IAL is a diachronic phenomenon has been substantiated by Misra (1967). According to Ohala (1983) the deletion of schwas is more frequent in casual and fast speech compared to formal and slower ones. It can be inferred from these facts that the motivation behind schwa deletion is faster communication through minimization of syllables (Tranel 1999).</Paragraph>
    <Paragraph position="1"> Some recent works on mathematical and simulation based modelling of language evolution (Boer, 2000; Cangelosi and Parisi, 2002; Nowak et al, 2002) suggests that several features of languages emerge due to some basic cognitive and articulatory factors. These models assume a) ease of articulation, b) ease of learning, and c) acoustic distinctiveness as the primary driving forces behind language evolution. The three forces operate simultaneously over the language in order to maximize the rate of successful communication in terms of time and effort spent by the language users to generate, understand and learn the language. Thus, language can be modelled as a multi-objective optimization system, where the optimization criteria are * Minimization of effort (in terms of energy and time spent while conveying a piece of information) * Minimization of learning time and effort * Minimization of probability of misunderstanding (in the sense of confusing one word with another) These three criteria are mutually contradictory and therefore there exists no global optimum. Let us examine the phenomenon of schwa deletion under this multi-objective optimization model for language evolution. When a vowel is deleted from a word the number of syllables reduces by one. For example, in Table 1, for the second word, Sanskrit and Bengali have three syllables, whereas due to the deletion of a schwa, the Hindi pronunciation has only two syllables. Reduction of syllables implies shorter time for pronunciation of a word, and hence faster communication. However, deletion of schwas in certain contexts might result in a consonant cluster which the native speakers find very difficult or impossible to pronounce. This beats the very purpose of schwa deletion, i.e. the minimization of effort of articulation and therefore, is unacceptable. The second condition for the rule proposed by Ohala (section 2) refers to this constraint.</Paragraph>
    <Paragraph position="2"> There are contexts where deletion of schwa would not give rise to inadmissible consonant clusters. For example, in the Hindi/Bengali word pari (fairy, /p@ ri/ in Hindi), if the first schwa is deleted, the pronunciation would be /pri/, which does not violate the phonotactic constraints of the languages. The schwa, however, is not deleted, because /p@ ri/ and /pri/ are too distinct from each other to be interpreted as the same word.</Paragraph>
    <Paragraph position="3"> Moreover, /pri/ is closer to other Hindi words like priya (favorite, /prij@ /). In this case, the deletion of schwa reduces the acoustic distinctiveness of the word from other words in the lexicon, which increases the probability of misunderstanding, and hence the schwa might not be deleted in such a context.</Paragraph>
  </Section>
  <Section position="5" start_page="1" end_page="3" type="metho">
    <SectionTitle>
4 Computational framework
</SectionTitle>
    <Paragraph position="0"> We propose the following diachronic explanation for schwa deletion in IAL.</Paragraph>
    <Paragraph position="1"> In old IAL none of the schwas are deleted. The modern IAL use the script and spelling conventions similar to Sanskrit. Due to a higher evolutionary pressure on the spoken forms of the languages than on the written forms, schwas are deleted in the pronunciation, but are still present in the graphemic forms. The deletion is a slow diachronic phenomenon, where in order to communicate faster, initially the speakers unintentionally deleted the schwas. Only those deletions were acceptable that did not lead to a syllable structure which was too difficult to pronounce, learn or understand for the native speakers. Gradually, the pattern of deletion spread across the population and over the different items in the lexicon.</Paragraph>
    <Paragraph position="2"> In this section, we describe a computational framework for modelling the aforementioned hypothesis based on the three optimization criteria stated in the last section. The aim of the proposed framework is not to validate the hypothesis through micro-simulation (Cangelosi and Parisi, 2002); rather it tries to predict the schwa deletion pattern based on the optimizations that might have affected the deletion of schwas diachronically. In the next section, we present an efficient algorithm for schwa deletion in IAL, which can be automatically constructed from this model, without the help of any other evidence.</Paragraph>
    <Section position="1" start_page="1" end_page="3" type="sub_section">
      <SectionTitle>
4.1 Basic definitions
</SectionTitle>
      <Paragraph position="0"> All the unexplained symbols used below stand for their usual meaning in the context of formal language theory. Please refer to (Hopcroft and Ullman, 1979) for details.</Paragraph>
      <Paragraph position="1">  is the default mapping of the graphemes to the phonemes. This oversimplification is made here for two reasons. First, since IAL use a phonetic script, this in general is true  and second, this assumption does not have any affect on the schwa deletion algorithm.</Paragraph>
      <Paragraph position="2"> A word w is defined as a 2-tuple &lt;w</Paragraph>
      <Paragraph position="4"> A lexicon L is the union of all the valid words w of a language. A grapheme-to-phoneme converter is defined as a function F</Paragraph>
    </Section>
    <Section position="2" start_page="3" end_page="3" type="sub_section">
      <SectionTitle>
4.2 Phonotactic constraints
</SectionTitle>
      <Paragraph position="0"> In order to model the ease of articulation, we start with the modelling of phonotactic constraints.</Paragraph>
      <Paragraph position="1"> A consonant cluster is a string of the form C</Paragraph>
      <Paragraph position="3"> Phonotactic constraints restrict the presence of some of the consonant clusters in the phonetic representation of a word (w p ). At the most generic level we can think of a consonant cluster ranking (CCR) function, where N is the set of natural  such that a consonant cluster x [?] C</Paragraph>
      <Paragraph position="5"> the language if and only if</Paragraph>
      <Paragraph position="7"> We define two special variants of CCR, O_CCR and C_CCR , which ranks the admissibility of the consonant clusters at the onset and coda positions respectively. The definition is similar to that of  This assumption is not strictly valid since a cluster of consonant might be mapped to a single consonant or a different cluster. The sonority hierarchy (Vennemann, 1988) and markedness conditions (Kager, 1999) along with the physiology of the articulatory mechanism point towards the existence of a language independent ranking function as hypothesized above. However, there might be accidental gaps in the list of admissible consonant clusters of a language (Ohala, 1983), which can not be explained on the basis of CCR alone. Therefore, we define a Boolean function ADM that tell us about the admissibility of consonant clusters in a language. ADM: C</Paragraph>
      <Paragraph position="9"> In general, we can derive this function from  However, we might have to forcefully convert some values to 0 due to accidental gaps.</Paragraph>
    </Section>
    <Section position="3" start_page="3" end_page="3" type="sub_section">
      <SectionTitle>
4.3 Syllable and Syllabification
</SectionTitle>
      <Paragraph position="0"> We define a syllable s as a regular expression, with the assumption that the nucleus contains a single vowel. Thus,  such that the effort of articulation and learning are minimum. We model the effort of articulation using a syllable ranking function SR, which is similar to</Paragraph>
      <Paragraph position="2"> SR is mainly dependent on the structure of the syllable. We enumerate the first few terms of the function SR.</Paragraph>
      <Paragraph position="4"> For all other possible syllable structures s',</Paragraph>
      <Paragraph position="6"> This means that if either the coda or the onset of a syllable is inadmissible, then the ranking function maps the syllable to the highest possible rank, represented symbolically by the infinity ([?]).</Paragraph>
      <Paragraph position="7"> onset and coda are projection functions that project the longest valid prefix and suffix of a syllable respectively that are elements of C p *.</Paragraph>
      <Paragraph position="8"> We define a syllabification to be valid if all the syllables are valid (i.e. strings of the form</Paragraph>
      <Paragraph position="10"> *) and every symbol in the word is a part of one and only one syllable in the syllabification. We can define a partial ordering, [?] s, among the possible valid syllabifications of a given word based on SR p such that the syllabification with smaller number of high ranked syllables is preferred to one that has more hard (high ranked) syllables. Now we define SYL (w p ) as the set of all  The definitions of syllable and syllabification are motivated by the markedness conditions (Kager, 1999) and experimental results on child language acquisition (MacNeilage and Davis, 2000), that show that some syllables and syllabifications are easier to learn and pronounce than others.</Paragraph>
    </Section>
    <Section position="4" start_page="3" end_page="3" type="sub_section">
      <SectionTitle>
4.4 Acoustic distinctiveness constraints
</SectionTitle>
      <Paragraph position="0"> Perceptual experiments show that speakers always articulate the onset of the syllables more clearly and correctly compared to the articulations of the vowel and the coda (Fosler-Lussier et al, 1999; Greenberg, 1999). Therefore, it is likely that the hearer distinguish between syllables by paying more weight to the onset than to the coda. A continuous distance metric D s might be defined based on these experimental results, such that the probability of confusion (interpreting one syllable as another) between two syllables s and s' increases as the value of D s (s , s') decreases. We can further define an acoustic distance function D</Paragraph>
      <Paragraph position="2"> , which measures the probability of confusion between two arbitrary words in the phonetic domain.</Paragraph>
      <Paragraph position="3"> In the case of schwa deletion, however, we want the acoustic distance between the ideal pronunciation (without any schwa deletion) and the normal pronunciation (with schwa deletion) to be smaller, so that the word is not confused with other words in the lexicon. Formally, for the graphemic representation of a word w  is the maximum allowable distance and '.' is the concatenation operator. Rather than modelling this as an optimization criterion, we reformulate this as a constraint. The simplification in this case serves our purpose. We define, where x [?] C</Paragraph>
      <Paragraph position="5"> For all other cases D s is infinity ([?]), unless the two syllables are identical. (4c) (4a) allows the deletion of a schwa from an open syllable; (4b) allows the concatenation of a consonant at the coda position. This is motivated by the fact that coda has least distinctiveness (Greenberg, 1999). (4c) restricts any change at the onset of a syllable or the vowels other than schwa. On the basis of D</Paragraph>
      <Paragraph position="7"> possible gaps (ph or null syllables) such that for all the corresponding pairs of syllable taken from the two sequences, the acoustic distinctiveness (D s ) between them is 0. Thus, only operations allowed are deletion of a schwa and addition of a consonant at the coda position. Anything else is forbidden for the sake of acoustic distinctiveness.</Paragraph>
      <Paragraph position="8"> We conclude this section by summarizing below the salient features of the model by comparing it with the optimization criteria stated in section 3. * The functions SR, CCR and its variants that rank the phonotactic constraints is a measure of the effort of articulation, learning and the probability of misunderstanding. Therefore we want to minimize it. However, it has been modelled as a constraint (ADM).</Paragraph>
      <Paragraph position="9"> * The function SYL is so defined that the efforts of articulation and learning are minimized.</Paragraph>
      <Paragraph position="11"> models the acoustic distinctiveness i.e.</Paragraph>
      <Paragraph position="12"> the criterion 3c, but it has been reformulated as a constraint as well.</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="3" end_page="3" type="metho">
    <SectionTitle>
5 The algorithm
</SectionTitle>
    <Paragraph position="0"> We want to define F g2p for a language given</Paragraph>
    <Paragraph position="2"> should be such that it enables faster communication by minimization of syllables by deletion of schwa.</Paragraph>
    <Section position="1" start_page="3" end_page="3" type="sub_section">
      <SectionTitle>
5.1 Formal definition
</SectionTitle>
      <Paragraph position="0"> in the next syllable  3.3 else NO syllabification is possible 4. If there is one or no consonant between the current vowel and the next vowel, terminate the current syllable and begin the next syllable 5. Continue from step 2 till there are symbols not included in any syllable.</Paragraph>
      <Paragraph position="1"> end procedure</Paragraph>
    </Section>
    <Section position="2" start_page="3" end_page="3" type="sub_section">
      <SectionTitle>
5.2 A greedy strategy
</SectionTitle>
      <Paragraph position="0"> Figure 1 describes a linear time algorithm for syllabification (SYL) that conforms to the definition provided in section 4.3. This uses the fact that the maximum length of allowable consonant clusters for IAL is three. After syllabification of w p , we try to greedily delete the schwas so that the constraints specified by 4a, 4b and 4c are not violated. 4a states that only a schwa which is a part of an open syllable (ca, where c [?] C p ) can be deleted and 4b states that after schwa deletion, the consonant c is appended to the coda of the previous syllable. Therefore, both of them together imply schwas in two consecutive syllables cannot be deleted. Along with that, the following constraints can also be derived from the D w constraints (the reasons are omitted due to space constraints): R1. Schwa of the first syllable cannot be deleted R2. Schwa cannot be deleted before a consonant cluster.</Paragraph>
      <Paragraph position="1"> R3. The word final schwa can always be deleted unless the appending of the penultimate consonant to the previous syllable results in an inadmissible cluster.</Paragraph>
      <Paragraph position="2"> R4. For Bengali, which does not allow complex codas, schwas cannot be deleted after consonant clusters.</Paragraph>
      <Paragraph position="3"> R5. A schwa followed by a vowel cannot be deleted.</Paragraph>
      <Paragraph position="4">  ' using procedure SYL 3. Using rules R1 to R6 and ADM constraints mark the schwas which cannot be deleted as F 4. While traversing the word from right to left 4.1 Delete a schwa if it is not marked F 4.2 Appended the dangling consonant to the coda of the adjacent syllable (to the left) 4.3 If the adjacent syllable (to the left) has a schwa which is unmarked, mark it F 4.4 Go to 4.1 if there are more schwas to the left of the current position.</Paragraph>
      <Paragraph position="5"> 5. At the end of step 4 we get the syllabified string of  We have the following rule that cannot be captured by the constraints: R6. Schwa following a y (pronounced as /j/) cannot be deleted if it is preceded by a high vowel because /j/ is a glide from high vowel to a low/medium vowel (schwa), deletion of schwa would make the presence of the glide imperceptible.</Paragraph>
      <Paragraph position="6"> This rule could have been captured by the D s constraints but we state it here as a separate rule for the sake of simplicity. Figure 2 describes an algorithm for schwa deletion using the rules above. It is easy to see that the time complexity of the algorithm is O(|w g |). Due to limited space, we omit the proof that the algorithm for F g2p indeed minimizes the number of syllables without violating the constraints specified by ADM and D w .</Paragraph>
      <Paragraph position="7"> However, there might be more than one (precisely 2) possible solutions and in that case the algorithm chooses one of the solutions on the basis of the direction of traversal at step 4. The right to left traversal gives better results (as has been confirmed by Ohala, 1983) because the duration of syllables reduces towards the end of the word and hence the tendency to delete schwas at the word final position increases.</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>