The movements of muscled animals are hierarchical and spatiotemporally complex. I will try to show in this talk that a dynamical systems approach to motor control as spatiotemporal pattern formation allows us to begin to understand common signatures of the hierarchical nature of movement in animals as simple as the 1-muscled C. elegans worm and as complex as the many-muscled octopus. It will further be argued that we may be able to deduce some important properties of gestures and syllables in speech production, if we approach the linguistic task of the tongue from a pattern formation perspective.

Our topic is the discrete unit of speech production, the gesture. We focus on the description of the single gesture and not on the coordination of two or more gestures. This presents the challenge of identifying kinematic properties of vocal tract events which are ascribable to the single gesture and not to two or more coproduced gestures. The talk accomplishes three things. First, we define the notion of the intrinsic dynamics of a gesture. Second, we present an argument that the intrinsic dynamics is observable in a controlled kinematic data-set. Third, we propose a model of the intrinsic gestural dynamics which a articulographic data-set supports.

Examples of nonlinearities between articulation and acoustics and between acoustics and perception abound. These nonlinearities allow communication to occur despite people having different vocal tracts with different resonance capabilities and despite much variation in the acoustic signal across speakers, emphasis, background noise, etc. Yet context and the listener’s expectations often strongly influence what is perceived; speech perception is dynamic, influenced by multiple factors that change slowly or quickly. In this talk, I will discuss a selection of dynamical models in speech categorization, imitation, and learning, and consider a dynamical interpretation of the neural activity during communication as a “linking hypothesis” for speaking and listening.

It is known that the phonotactics of a number of languages cannot be accounted for by a strict interpretation of the sonority sequencing principle. In this talk I review cases of more or less extreme departure from the principle in a variety of languages, with a focus on Caucasian languages, which allow sonority reversals or plateaus. Based on experimental results, I discuss a possible view of sonority that refers to patterns of temporal coordination. I propose a possible interpretation of the sonority scale as it relates to articulatory timing.

What is easy to say, and what is difficult? The notion of “ease of articulation” has been controversial since Jakobson’s and Trubetzkoy’s times, especially because it lacks a sufficiently firm operational and empirical basis. Investigations of the speech motor system through functional brain imaging are usually based on implicit assumptions about what is cheap or expensive from the perspective of the motor effort involved in articulating the stimulus words used in an experiment.

One important aspect that must in my opinion be considered is that articulatory skills are subject to practice-related functional and neural plasticity mechanisms: motor patterns which may appear intrinsically difficult from some universal point-of-view can become easy through the extensive motor learning that occurs during the acquisition of a speaker’s native sound pattern. Therefore, ease-of-articulation in adults, if it exists, must be language-particular. Yet, differences that may exist between the articulo-motor demands of particular syllables or words in a speaker’s language exhibit a strong ceiling effect in adults because they are all “easy to say”. Hence, healthy adult speakers are not a particularly suitable source to measure the relative motor costs of different syllables or words within a given language, for instance through chronometric experiments or functional brain imaging.

In our work we consider a particular neurological condition to provide clues to the motor requirements of different sound patterns in a language. Lesions to anterior perisylvian and sub-sylvian cortex of the left hemisphere may cause a severe articulation problem termed apraxia of speech. I will briefly point out why we consider apraxia of speech as a suitable model to investigate the architecture of the motor skills involved in adult speech. The major part of the talk will then deal with the factors that influence accuracy of articulation in apraxia of speech, and I will present a non-linear model which integrates these factors to predict the likelihood of apraxic speech errors from the phonological make-up of a word. I will conclude with an outlook of how these data can be interpreted and where this research may lead us to in the future.

Dysarthria in Parkinson’s disease (PD) is generally described by a monotony of pitch and loudness, reduced stress, variable rate, imprecise consonants, and a breathy and harsh voice. Among surgical treatments, deep brain stimulation (DBS) of the subthalamic nucleus (STN) has proven efficiency in alleviating the cardinal motor symptoms of PD; it is able also to notably improve some motor components of speech, while unfortunately intelligibility generally decreases after surgery. Results available often confirm the lower degree of speech improvement following STN stimulation compared with the total motor state; the surgery can even lead to stimulation-induced speech impairment in some patients, sometimes exacerbating a pre-existent dysarthria. Current diffusion has been proposed as a possible explanation for such speech impairment side-effects (e.g., current diffusion towards both the spinal (Pinto et al., 2005) and the cerebello-thalamo-cortical (Astrom et al., 2010) tracts). Thus, management of dysarthria in PD is still challenging for the clinician. In this context, functional brain imaging can address issues to better understand the particular physiopathology of PD dysarthria.

Deep brain stimulation of the ventral intermediate nucleus (VIM) of the thalamus is a powerful treatment option for medication-refractory essential tremor (ET). In this talk I will give a brief overview on the clinical picture of ET, the current understanding of how VIM-DBS suppresses tremor and why certain stimulation induced side-effects occur. Here, I will focus on stimulation-induced dysarthria and will present data on how electrical field shaping might reduce side effects without loosing the effect on tremor control. Finally, I would like to discuss the pathophysiological understanding of VIM-DBS induced dysarthria in ET.

The present study deals with acoustics and kinematics of speech in essential tremor (ET) patients treated with deep brain stimulation (DBS). ET patients were recorded with a 3-D Electromagnetic Articulograph with stimulation ON and OFF. In a first study, we recorded fast syllable repetition tasks (DDK) and measured intra-gestural coordination patterns. In a second study, we recorded target words with simple and complex onsets and measured the inter-gestural coordination patterns of oral constriction gestures. The findings reveal a detrimental decrease in stiffness of oral gestures in DBS-on condition leading to imprecise articulation in the production of stop consonants. Further, ET patients show deviant coordination patterns in syllable production. The timing deficits are related to the loss of learnt syllable internal coupling relations.

Italian shows both singleton and geminate consonants which have been investigated also in relation with the coordination of vowel and consonant gestures. Though the first analyses suggested a possible description through Ohman’s (1967) Vowel-to-Vowel model (Smith 1995), following studies did not confirm the suitability of the abovementioned model, which predicts vowels would be produced independently of intervening consonants: according to this model, alternating singleton/geminate consonants would not produce any modification to the flanking vowels (at least to the duration of the transition between the two). For instance, in relation to the presence of singleton and geminate consonants Gili Fivela & Zmarich (2007) observe the variable duration of the vowel-to-vowel gesture, the only partially constant timing of maximum consonantal constriction, and the anticipation of the geminate consonantal gesture in the preceding vowel, with no delay in the following vowel. Similar data were then argued to be associated with a sort of hybridization of Ohman’s (1967) and Browman & Goldstein’s (1990) Vowel-and-Consonant model.

Previous works on Parkinson and control speakers suggest a stable pattern for the distinction between singletons and geminates, confirming acoustic and kinematic parameters known in literature, moreover accompanied by a clear effect on the tongue gesture coproduced in the phase of germination (cfr. Bertinetto 1983, Gili Fivela & Zmarich 2007, Gili Fivela et al. 2014; Iraci et al. submitted). Nevertheless, they also show that Parkinson speakers show alterations in the expected realization of the articulators’ displacement magnitude (Ackermann & Ziegler 1991; Skodda 2011, 2012; Wong 2010, 2011), sometimes associated to phasing differences (Gili Fivela et al. 2014).

In this talk I will discuss the stability of this pattern through opposite vocalic contexts and its realization by pathological and control speakers, focusing on the inter-articulator programming also in relation with the alterations in articulators’ displacement magnitude.

Explanations of motor area activation during speech perception (e.g., D’Ausilio et al., 2011) are often couched in underspecified notions that mirror-neuron-like mechanisms support analysis-by-synthesis; true progress requires concrete, mechanistic theories. I will report on first steps towards developing a computational model of the co-development of speech and reading by a research team based at Haskins Labs. We are extending models based on attractor networks to link models of speech perception and reading to incorporate speech production to aid the development of a theory of co-development of speech and reading. In this framework, even distant information processing units have potential reciprocal influences during development and processing. This leads logically to predictions that articulatory knowledge should be especially important during development, and under noisy/ambiguous conditions (just as attractor network models of reading predict greater reliance on semantic pathways for unusual spelling-sound patterns or under noisy/ambiguous conditions). Our approach is inspired by and extends a model of speech perception and production reported by Plaut and Kello (1999) by adding a much more realistic, implemented model of articulation-acoustics (TADA; Nam et al., 2004), more realistic semantic representations, and integrated orthographic units. This model (still under development) will allow us to investigate computationally how perceptual and orthographic processes and representations interactively change when linked to production (and vice-versa) – a vital step towards understanding the role of articulation in perception and reading, and reciprocal influences among all three domains. I will review the rationale and empirical basis for the model and report on our progress thus far. 

I present an argument for extending the dynamical approach from gestural representations to principles in grammars using a classic phonological phenomenon. I highlight the key similarities and differences between dynamics in grammars and standard symbolic grammars. I then return to representations by presenting an extension of the dynamical approach to the assembly of phonological form using a task where participants listen to various stimuli as they utter simple syllables; this task zooms in the assembly of phonological representations at Catford’s microchronic time scale. Dynamical principles pervade across the different examples. Why then do grammars seem mostly symbolic? I argue that this fact follows as a consequence of two properties, the hierarchical nature of cognition as emphasized in Smolensky (1988, 2006) and the allied notion of near-decomposability of time scales as promoted in work by Simon (1973) and Rosen (1969).

The present study explores prosodic strengthening in Korean and compare the results with those of English in an effort to understand how languages may differ in the way prosodic strengthening is phonetically realized. Results indicate that (1) the transboundary tongue movement is faster in Korean (but slower in English), and (2) the lip opening gesture is faster/larger in Korean (but slower in English). These results demonstrate some degree of inseparability of boundary- versus prominence-driven strengthening in Korean, which is remarkably different from prosodic strengthening patterns observed in English. The cross-linguistic difference is interpreted as stemming from different prosodic systems of the languages. With no functional demands that may come from the lexical stress system, Korean appears to have more freedom to strengthen articulation at prosodic junctures in a way that is comparable to prominence marking. This is consistent with the observation that focus marking in Korean is more likely accompanied by prosodic phrasing headed by the focused word in the domain-initial position. It is therefore proposed that prominence marking is modulated by boundary marking in Korean, in such a way that domain-initial strengthening is accompanied by some degree of prominence. The results will be discussed in terms of their implications regarding how dynamical theories may account for the extent to which articulatory variation due to prosodic strengthening is cross-linguistically applicable versus language-specifically attuned.

In this talk we show that, in consonant final loan words and foreign names in Italian, the tune can condition adjustments to the text in terms of the insertion and acoustic prominence of schwa (fred /frɛddə/). However, we also show that this insertion is probabilistically distributed and dependent on other factors too, such as the metrical structure of the target word and the laryngeal specification of the preceding consonant.

We also show adjustments to the text in terms of schwa in a genetically unrelated language, Tashlhiyt Berber. In this language too, schwa appears to be conditioned by the requirement to express functionally relevant tonal movements, although the position of this schwa is not restricted to word final position.

Thus, in both languages, intonational tones may not be the only factors involved, but they clearly play a considerable role in determining the restruct­uring of the textual material with which they are associated. In this sense we can conclude that the tune plays a significant role in driving the text.

This study presents articulatory data on the durational characteristics of Russian and German C1C2V onset clusters. We show that the same segmental sequence has very different durational and coarticulatory patterns in the two languages. Specifically, Russian contrasts with German in that consonant plateaus overlap relatively less than in German, with Russian systematically permitting transitional schwas between consonants. However, plateau-lag measures are insufficient to capture the fundamental difference in coarticulation between the two languages. While in Russian the plateau lag between consonants is greater than in German, movement onset of C2 is significantly earlier. That is, when movement onset is considered, Russian shows much greater consonant coarticulation compared to German. Moreover, there is a systematic positive correlation between plateau lag and movement onset of C2 relative to C1 for the German data. This is not the case for Russian for which there is only a small negative correlation reflecting the combination of early movement onset for C2 with a considerable C1-C2 plateau lag. Correspondingly C2 closure formation is, for the identical segments, systematically slower in Russian compared to German. The pattern described in previous research that the cluster-internal member (C2) is of shorter duration than the external one (C1) holds in Russian for plateau duration only. When considering constriction formation C2 is longer than C1. German shows almost the reverse pattern: C2 has a longer plateau duration than C1, but about the same closure formation duration as C1. Our data underscore the language-specific nature not only of the relative timing of C1 and C2, but also of the intrinsic consonant kinematics governing closure formation and plateau duration. We discuss problems with current dynamic models of duration in the light of these language-specific patterns.

A model of speech timing has been proposed in recent work [3], in which individual speech gestures are associated with ”planning oscillators” that are coupled to one another in coupling graphs that embody the basic syllable structures of a language [3]. Two sources of evidence for this model of timing have been discussed: data from gestural intrusion speech errors suggesting a role of oscillator mode-locking in speech production planning [4] and a variety of temporal phenomena that can be understood as consequences of the topology of the proposed syllable-level coupling graphs. Two recent findings on birdsong will be discussed that suggest a similar style of temporal regulation: evidence for mode-locking transitions of syrinx gestures under conditions of rate change [2] and evidence of pulses in the premotor cortex that are with time-locked with extrema (onsets and offsets) of syrinx gestures during singing [1].

[1] Amador, A., Perl, Y., Mindlin, G. & Margoliash, D. (2012). Elemental gesture dynamics are encoded by song premotor cortical neurons. Nature, 495, 59-64.

[2] Goldin, M. & Mindlin, G. (2003). Evidence and control of bifurcations in a respiratory system. Chaos, 23, 043138-1 – 043138-6.

[3] Goldstein, L., Byrd, D., & Saltzman, E. (2006). The role of vocal tract gestural action units in understanding the evolution of phonology. In M. A. Arbib (Ed.), Action to Language via the Mirror Neuron System. (pp. 215-48). Cambridge University Press.

[4] Goldstein, L., Pouplier, M., Chen, L., Saltzman, E., & Byrd, D. (2007). Dynamic action units slip in speech production errors. Cognition, 103(3), 386-412.