<?xml version="1.0" standalone="yes"?>
<Paper uid="W05-0618">
  <Title>Beyond the Pipeline: Discrete Optimization in NLP</Title>
  <Section position="9" start_page="141" end_page="142" type="concl">
    <SectionTitle>
6 Conclusions
</SectionTitle>
    <Paragraph position="0"> In this paper we argued that pipeline architectures in NLP can be successfully replaced by optimization models which are better suited to handling correlated tasks. The ILP formulation that we proposed extends the classi cation paradigm already established in NLP and is general enough to accommodate various kinds of tasks, given the right kind of data. We applied our model in an NLG application. The results we obtained show that discrete  optimization eliminates some limitations of sequential processing, and we believe that it can be successfully applied in other areas of NLP. We view our work as an extension to Roth &amp; Yih (2004) in two important aspects. We experiment with a larger number of tasks having a varying number of labels.</Paragraph>
    <Paragraph position="1"> To lower the complexity of the models, we apply correlation tests, which rule out pairs of unrelated tasks. We also use stochastic constraints, which are application-independent, and for any pair of tasks can be obtained from the data.</Paragraph>
    <Paragraph position="2"> A similar argument against sequential modularization in NLP applications was raised by van den Bosch et al. (1998) in the context of word pronunciation learning. This mapping between words and their phonemic transcriptions traditionally assumes a number of intermediate stages such as morphological segmentation, graphemic parsing, grapheme-phoneme conversion, syllabi cation and stress assignment. The authors report an increase in generalization accuracy when the the modular decomposition is abandoned (i.e. the tasks of conversion to phonemes and stress assignment get con ated and the other intermediate tasks are skipped). It is interesting to note that a similar dependence on the intermediate abstraction levels is present in such applications as parsing and semantic role labelling, which both assume POS tagging and chunking as their preceding stages.</Paragraph>
    <Paragraph position="3"> Currently we are working on a uniform data format that would allow to represent different NLP applications as multi-task optimization problems. We are planning to release a task-independent Java API that would solve such problems. We want to use this generic model for building NLP modules that traditionally are implemented sequentially.</Paragraph>
    <Paragraph position="4"> Acknowledgements: The work presented here has been funded by the Klaus Tschira Foundation, Heidelberg, Germany. The rst author receives a scholarship from KTF (09.001.2004).</Paragraph>
  </Section>
class="xml-element"></Paper>