The following links point to my public profiles on the research portals I use:
Below, is a list of my publications (with some relevant links) that I try to keep up to date.
Publications
2015
"Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples and Cast Studies", Kelleher, J.D., Mac Namee, B and D'Arcy. MIT Press.
[abstract]
[bib]
[Book Website]
[MIT Book Website]
Machine learning is often used to build predictive models by extracting patterns from large datasets. These models are used in predictive data analytics applications including price prediction, risk assessment, predicting customer behavior, and document classification. This introductory textbook offers a detailed and focused treatment of the most important machine learning approaches used in predictive data analytics, covering both theoretical concepts and practical applications. Technical and mathematical material is augmented with explanatory worked examples, and case studies illustrate the application of these models in the broader business context.
After discussing the trajectory from data to insight to decision, the book describes four approaches to machine learning: information-based learning, similarity-based learning, probability-based learning, and error-based learning. Each of these approaches is introduced by a nontechnical explanation of the underlying concept, followed by mathematical models and algorithms illustrated by detailed worked examples. Finally, the book considers techniques for evaluating prediction models and offers two case studies that describe specific data analytics projects through each phase of development, from formulating the business problem to implementation of the analytics solution. The book, informed by the authors’ many years of teaching machine learning, and working on predictive data analytics projects, is suitable for use by undergraduates in computer science, engineering, mathematics, or statistics; by graduate students in disciplines with applications for predictive data analytics; and as a reference for professionals.
[hide]
@book{kelleherelal:15,
author = {Kelleher, J.D. and Mac Namee, B. and D'Arcy, A.},
title = {Fundamentals of Machine Learning for Predictive Data Analytics: Algorithms, Worked Examples and Case Studies},
publishers = {MIT Press},
year = {2015},
}
[hide]
"The Recruitment of Passion and Community in the Service of Capital: Community Managers in the Digital Games Industry", Kerr, A. and Kelleher, J.D. Critical Studies in Media Communication
[abstract]
[bib]
[pdf]
Globalization and technical change have had a significant impact on work in the cultural industries. Online games are services that are networked, operate around the clock and require ongoing player and company input. The industry's content production networks are dispersed internationally, and many of its services are offered transnationally. These new services have generated new forms of work, like community management, which are often outsourced to near-to-market locations. Much of the research on media work is focused on high status creative occupations or the free labour of amateurs. This paper draws upon the findings of a content analysis of job advertisements and face-to-face interviews with community managers to examine the recruitment and work of community managers. Using scholarship on media production, media work and emotional labour this article argues that recruitment and organizational practices surrounding community managers appropriate passion, community, and experience in the service of capital, but also marginalize these workers. While community managers see themselves as creative workers, their visibility and creative autonomy are limited by organizational and workplace cultures.
[hide]
@article{kerrkelleher:15,
author = {Kerr, A. and Kelleher, J.D.},
title = {The Recruitment of Passion and Community in the Service of Capital: Community Managers in the Digital Games Industry},
journal = {Critical Studies in Media Communication},
year = {2015},
}
[hide]
"Changing perspective: Local Alignment of reference frames in dialogue", Dobnik, S., Howes, C. and Kelleher, J.D. Proceedings of the 19th Workshop on the Semantics and Pragmatics of Dialogue, pages 31--39.
[abstract]
[bib]
[pdf]
In this paper we examine how people
negotiate, interpret and repair the frame
of reference (FoR) in free dialogues dis-
cussing spatial scenes. We describe a pilot
study in which participants are given dif-
ferent perspectives of the same scene and
asked to locate several objects that are only
shown on one of their pictures. This task
requires participants to coordinate on FoR
in order to identify the missing objects.
Preliminary results indicate that conversa-
tional participants align locally on FoR but
do not converge on a global frame of refer-
ence. Misunderstandings lead to clarifica-
tion sequences in which participants shift
the FoR. These findings have implications
for situated dialogue systems.
[hide]
@inproceedings{dobniketal:15,
author = {Dobnik, S. and Howes, C. and Kelleher, J.D.},
title = {Changing perspective: Local Alignment of reference frames in dialogue},
booktitle = {Proceedings of the 19th Workshop on the Semantics and Pragmatics of Dialogue}
pages = {31--39}
year = {2015},
}
[hide]
"A Model for Attention-Driven Judgements in Type Theory with Records", Kelleher, J.D. and Dobnik, S. Proceedings of Interactive Meaning Construction A Workshop at IWCS 2015
[abstract]
[bib]
[pdf]
Recently, Type Theory with Records (TTR, (Cooper, 2012; Cooper et al., 2014)) has been proposed
as a formal representational framework and a semantic model for embodied agents participating in situated dialogues (Dobnik et al., 2014). Although TTR has many potential advantages as a semantic model for embodied agents, one problem it faces is the combinatorial explosion of types that is implicit in the framework and is due the fact that new types can be created or learned by an agent dynamically. Types are intensional which means that a given situation in the world may be assigned more than one record type. A sensory reading of a particular situation in the world involving spatial arrangement of objects may be assigned several record types of spatial relations simultaneously, for example Left, Near, At, Behind, etc. TTR also incorporates the notion of sub-typing which allows comparison of types. A situation judged as being of a particular record type may also be judged of potentially infinite number of its sub-types: a situation of type Table-Left-Chair is also of type Table and Left, etc. In this presentation we argue that agents need (i) a judgement control mechanism and (ii) a method for organising their type inventory. For (i) we propose the Load Theory of selective attention and cognitive control (Lavie et al., 2004) to be a suitable candidate. For the requirement (ii) above we propose that agents organise their type inventory into subsets or bundles of types that are represented as cognitive states. We propose that a (probabilistic) POMDP framework (Partially Observable Markov Decision Processes, (Kaelbling et al., 1998)) provides a useful mathematical model for implementations of a control structure for judgements in an embodied agent/robot using TTR that has learned or been given by its designer a number of cognitive states.
[hide]
@inproceedings{kelleherdobnik:15,
author = {Kelleher, J.D. and Dobnik, S.},
title = {A Model for Attention-Driven Judgements in Type Theory with Records},
booktitle = {Proceedings of Interactive Meaning Construction, a workshop at IWCS 2015}
pages = {17--18}
year = {2015},
}
[hide]
2014
"Evaluation of a Substitution Method for Idiom Transformation in Statistical Machine Translation ", Salton, G.D., Ross, R.J. and Kelleher, J.D. Proceedings of the 10th Workshop on Multiword Expressions (MWE 2014) at the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Gothenberg, Sweden, 2014.
[abstract]
[bib]
[pdf]
We evaluate a substitution based technique for improving Statistical Machine Translation performance on idiomatic multiword expressions. The method operates by performing substitution on the original idiom with its literal meaning before translation, with a second substitution step replacing literal meanings with idioms following translation. We detail our approach, outline our implementation and provide an evaluation of the method for the language pair English/Brazilian-Portuguese. Our results show improvements in translation accuracy on sentences containing either morphosyntactically constrained or unconstrained idioms. We discuss the consequences of our results and outline potential extensions to this process.
[hide]
@inproceedings{saltonMWEetal:15,
author = {Salton, G.D. and Ross, R.J. and Kelleher, J.D.},
title = {Evaluation of a Substitution Method for Idiom Transformation in Statistical Machine Translation},
booktitle = {Proceedings of the 11th Workshop on Mutliword Expressions (MWE 2014) at the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL)}
address = {Gothenberg, Sweden}
year = {2014},
}
[hide]
"An Empirical Study of the Impact of Idioms on Phrase Based Statistical Machine Translation of English to Brazilian-Portuguese", Salton, G.D., Ross, R.J. and Kelleher, J.D. In Proceedings of the Third Workshop on Hybrid Approaches to Translation (HyTra) at the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL), Gothenburg, Sweden, 2014.
[abstract]
[bib]
[pdf]
This paper describes an experiment to evaluate the impact of idioms on Statis-tical
Machine Translation (SMT) process using the language pair English/Brazilian-Portuguese.
Our results show that on sen-tences containing idioms a standard SMT system achieves
about half the BLEU score of the same system when applied to sentences that do not
contain idioms. We also provide a short error analysis and out-line our planned work to
overcome this limitation.
[hide]
@inproceedings{saltonMWEetal:15,
author = {Salton, G.D. and Ross, R.J. and Kelleher, J.D.},
title = {An Empirical Study of the Impact of Idioms on Phrase Based Statistical Machine Translation of English to Brazilian-Portuguese},
booktitle = {Proceedings of the Third Workshop on Hybrid Approaches to Translation (HyTra) at the 14th Conference of the European Chapter of the Association for Computational Linguistics (EACL)}
address = {Gothenberg, Sweden}
year = {2014},
}
[hide]
"Using the Situational Context to Resolve Frame of Reference Ambiguity in Route Descriptions", Ross, R.J. and Kelleher, J.D. In Proceedings of the Second Workshop on Action, Perception and Language (APL), Uppsala, Sweden 2014.
[abstract]
[bib]
[pdf]
[APL2 Website]
The application domain for this research is human-robot interaction, specifically the interpretation of route instructions by robots. Two features of spatial language that are of particular relevance to route instruction interpretation are the notions of perspective and spatial reference frame. This paper presents a computational model that addresses the problem of resolving frame of reference ambiguity in route descriptions. This model works by using non-linguistic information from the situational context.
[hide]
@inproceedings{saltonMWEetal:15,
author = {Ross, R.J. and Kelleher, J.D.},
title = {Using the Situational Context to Resolve Frame of Reference Ambiguity in Route Descriptions},
booktitle = {Proceedings of the Second Workshop on Action, Perception and Language (APL), Uppsala, Sweden 2014.}
address = {Uppsala, Sweden}
year = {2014},
}
[hide]
2013
Details for this year will be posted soon!
2012
Details for this year will be posted soon!
2011
Details for this year will be completed soon!
"The Effect of Occlusion on the Semantics of Projective Spatial Terms: A Case Study in Grounding Language in Perception", Kelleher, J.D., Ross, R., Mac Namee, B and Sloan, C. Cognitive Processing 12(1), 95-108.
[abstract]
[bib]
[pdf]
Although data-driven spatial template models provide a practical and cognitively motivated mechanism for characterizing spatial term meaning, the influence of perceptual rather than solely geometric and functional properties has yet to be systematically investigated. In light of this, in this paper we investigate the effects of the perceptual phenomenon of object occlusion on the semantics of projective terms. We did this by conducting a study to test whether object occlusion had a noticeable effect on the acceptance values assigned to projective terms with respect to a 2.5 dimensional visual stimulus. Based on the data collected a regression model was constructed and is presented. Subsequent analysis showed that the regression model which included the occlusion factor outperformed an adaptation of Regier and Carlson's well regarded AVS model for that same spatial configuration.
[hide]
@article{kelleherelal:11,
author = {Kelleher, J.D. and Ross, R. and Mac Namee, B. and Sloan, C.},
title = {The Effect of Occlusion on the Semantics of Projective Spatial Terms: A Case Study in Grounding Language in Perception},
journal = {Computational Processing},
volume = {12(1)}
pages = {95-108}
year = {2011},
}
[hide]
Kelleher, J.D. Visual Salience and the Other One. In Salience: Multidisciplinary Perspectives on its Function in Discourse, pages 205-228 (Eds) C. Chiarcos, B. Claus, M. Grabski. Mouton De Gruyter.
[abstract]
[bib]
[pdf]
This paper describes a salience based approach to visually situated reference
resolution. The framework uses the relationship between referential form and
preferred mode of interpretation as a basis for a weighted integration of linguistic
and visual salience scores for each entity in the multimodal context.
The resulting integrated salience scores are then used to rank the candidate
referents during the resolution process, with the candidate scoring the highest
selected as the referent. One advantage of this approach is that the resolution
process occurs within the full multimodal context, in so far as the referent is
selected from a full list of the objects in the multimodal context. As a result
situations where the intended target of the reference is erroneously excluded,
due to an individual assumption within the resolution process, are avoided.
[hide]
@InBook{tilsm:11,
author = {Kelleher, J.D.},
editor = {Christian Chiarcos and Berry Claus and Michael Grabski},
title = {Salience: Multidisciplinary Perspectives on its Functions in Discourse},
chapter = {Visual Salience and th Other One},
pages = {205--228},
year = {2011},
}
[hide]
2010
Visual Salience and Reference Resolution in Situated Dialogues: A Corpus-based Evaluation. Schutte, N., Kelleher, J.D. and Mac Namee, B. In Proc. of the AAAI Symposium on Dialog with Robots, Arlington, Virginia, USA. 11th - 13th Nov 2010.
[abstract]
[bib]
[workshop url]
Dialogues between humans and robots are necessarily situated. Exophoric references to objects in the shared visual context are very frequent in situated dialogues, for example when a human is verbally guiding a tele-operated mobile robot. We present an approach to automatically resolving exophoric referring expressions in a situated dialogue based on the visual salience of possible referents. We evaluate the effectiveness of this approach and a range of different salience metrics using data from the SCARE corpus which we have augmented with visual information. The results of our evaluation show that our computationally lightweight approach is successful, and so promising for use in human-robot dialogue systems.
[hide]
@InProceedings{schutteetal/etal:10,
author = {Schutte, N., Kelleher, J.D. and Mac Namee, B.},
title = {Visual Salience and Reference Resolution in Situated Dialogues: A Corpus-based Evaluation.},
booktitle = {In Proc. of the AAAI Symposium on Dialog with Robots, Arlington, Virginia, USA. 11th - 13th Nov 2010.},
year = {2010},
}
[hide]
Situating Spatial Templates for Human-Robot Interaction. Kelleher, J.D., Ross, R.J., Mac Namee, B. and Sloan, C. In Proc. of the AAAI Symposium on Dialog with Robots, Arlington, Virginia, USA. 11th - 13th Nov 2010.
[abstract]
[bib]
[workshop url]
People often refer to objects by describing the object's spatial location relative to another object. Due to their ubiquity in situated discourse, the ability to use 'locative expressions' is fundamental to human-robot dialogue systems. A key component of this ability are computational models of spatial term semantics. These models bridge the grounding gap between spatial language and sensor data. Within the Artificial Intelligence and Robotics communities, spatial template based accounts, such as the Attention Vector Sum model (Regier and Carlson, 2001), have found considerable application in mediating situated human-machine communication (Gorniak, 2004; Brenner et a., 2007; Kelleher and Costello, 2009).
Through empirical validation and computational application these template based models have proven their usefulness. We argue, however, that these models ignore important contextual features; resulting in their over-generalization and failure to account for actual usage in situated context. Such over-simplifications are a natural consequence of the experimental design taken in acquiring these models. That is, the data behind and hence the subsequent modelling of template based accounts used simplified scenes and reduced 2-dimensional survey based object configurations. While this is understandable given the original aims of these studies, we nevertheless believe that this is not sufficient justification for the direct application of idealized spatial templates to situated communication.
This critique of template based models is similar in spirit to critiques already put forward by a number of researchers: Coventry and Garrod (2004) have stressed the need to account for functional effects; Kelleher and Costello (2009) highlighted the need to account for the effects introduced by distractors. Here, we argue that the models must also be extended to incorporate perspective effects.
[hide]
@InProceedings{kelleheretal/etal:10b,
author = {Kelleher, J.D., Ross, R.J., Mac Namee, B. and Sloan, C.},
title = {Situating Spatial Templates for Human-Robot Interaction.},
booktitle = {In Proc. of the AAAI Symposium on Dialog with Robots, Arlington, Virginia, USA. 11th - 13th Nov 2010.},
year = {2010},
}
[hide]
Kelleher, J.D. and Ross, R.J. ''Topology in Composite Spatial Terms'', Poster presented at the International Conference on Spatial Cognition 2010, Mt. Hood/Portland, Oregon, USA, August 15-19.
[extended abstract pdf]
[poster pdf]
[conference url]
[bib]
@InProceedings{kelleher/ross:10,
author = {Kelleher, J.D. and Ross, R.J.},
title = {Topology in Composite Spatial Terms.},
booktitle = {Poster presented at the International Conference on Spatial Cognition 2010},
year = {2010},
}
[hide]
Ross, R., Hois, J. and Kelleher, J.D. (eds.) Proceedings of the Workshop on Computational Models of Spatial Language Interpretation at Spatial Cognition 2010 (COSLI-2010). August 15, Mt. Hood/Portland, Oregon, USA. CEUR-WS.org. ISSN 1613-0073.
[pdf]
[workshop url]
[bib]
@Proceedings{rossetal:10,
editor = {Ross, R., Hois, J. and Kelleher, J.D.},
title = {Proceedings of the Workshop on Computational Models of Spatial Language Interpretation at Spatial Cognition 2010},
address = {Mt. Hood/Portland, Oregon, USA},
month = {August 15},
year = {2010},
ISSN = {1613-0073},
publisher = {CEUR-WS.org},
url = {http://ceur-ws.org/Vol-620/}
}
[hide]
Kelleher, J.D., Mac Namee, B., van der Sluis, I. (eds.) Proceedings of the Sixth Internationl Natural Langauge Generation Conference (INLG 2010). 7th to 9th July, Trim, Co. Meath, Ireland.
[pdf]
[conference url]
[bib]
@Proceedings{kelleheretal:10,
editor = {Kelleher, J.D., Mac Namee, B. and van der Sluis, I.},
title = {Proceedings of the Sixth International Natural Language Generation Conference (INLG 2010)},
address = {Trim, Co. Meath, Ireland},
month = {July 7 to 9},
year = {2010},
publisher = {The Association of Computational Linguistics},
}
[hide]
Mark Dunne, Brian Mac Namee and John D. Kelleher, ''Intelligent Virtual Agent: Creating a Multi-Modal 3D Avatar Interface'', In Proceedings of the 23rd Conference on Computer Animation and Social Agents (CASA '10), Saint-Malo, France, May 31 - June 2.
[pdf]
[conference url]
2009
John D. Kelleher and Fintan J. Costello, "Applying Computational Models of Spatial Prepositions to Visually Situated Dialog", Computational Linguistics June 2009, Vol. 35, No. 2, Pages 271-306.
[pdf]
[abstract]
[bib]
This article describes the application of computational models of spatial prepositions to visually situated dialog systems. In these dialogs, spatial prepositions are important because people often use them to refer to entities in the visual context of a dialog. We first describe a generic architecture for a visually situated dialog system and highlight the interactions between the spatial cognition module, which provides the interface to the models of prepositional semantics, and the other components in the architecture. Following this, we present two new computational models of topological and projective spatial prepositions. The main novelty within these models is the fact that they account for the contextual effect which other distractor objects in a visual scene can have on the region described by a given preposition. We next present psycholinguistic tests evaluating our approach to distractor interference on prepositional semantics, and illustrate how these models are used for both interpretation and generation of prepositional expressions.
[hide]
@article{kelleher/costello:09,
author = {Kelleher, J.D. and Costello, F.},
title = {Applying Computational Models of Spatial Prepositions to Visually Situated Dialog},
journal = {Computational Linguistics},
volume = {35(2)}
pages = {271-306}
year = {2009},
}
[hide]
John D. Kelleher, Colm Sloan and Brian Mac Namee, "An Investigation into the Semantics of English Topological Prepositions", Cognitive Processing (Special Issue for ICSC) September 2009 Volumen 10 No 2. Pages 233-236.
[pdf]
[abstract]
[bib]
This paper describes a psycholinguistic experiment that investigates whether the applicability of the topological spatial prepositions ''at'', ''on'' or ''in'' to describe the spatial configuration between two objects is related to the topological relationships between objects being described.
[hide]
@article{kelleheretal:09,
author = {Kelleher, J.D., Sloan, C. and Mac Namee, B.},
title = {An Investigation into the Semantics of English Topological Prepositions},
journal = {Cognitive Processing (Special Issue for ICSC)},
volume = {10(2)}
pages = {233-236}
year = {2009},
}
[hide]
Brian Mac Namee and John D. Kelleher, ''Stepping Off the Stage'', In Proceedings of the 22nd Annual Conference on Computer Animation and Social Agents (CASA '09).
Mark Dunne, Brian Mac Namee and John D. Kelleher, ''Intelligent Virtual Agent: Creating a Multi-Modal 3D Avatar Interface'', In Proceedings of the 9th Annual Information Technology & Telecommunication Conference (IT&T '09).
2008
Brian Mac Namee, John D. Kelleher and Sarah Jane Delany, ''Medical Language Processing for Patient Diagnosis Using Text Classification and Negation Labelling'', In Proceedings of the Second i2b2 Shared-Task Workshop on Challenges in Natural Language Processing for Clinical Data, American Medical Informatics Association Annual conference (AMIA '08).
John D. Kelleher and Brian Mac Namee, ''Referring Expression Generation Challenge 2008 DIT System Descriptions (DIT-FBI, DIT-TVAS, DIT-CBSR, DIT-RBR, DIT-FBI-CBSR, DIT-TVAS-RBR)'', In Proceedings of the 5th International Natural Language Generation Conference (INLG-08).
2007
Kelleher, J.D. DIT - Frequency Based Incremental Attribute Selection for GRE. In Proceedings of the MT Summit XI Workshop Using Corpora for Natural Language Generation: Language Generation and Machine Translation (UCNLG+MT), Pages 90-92. Blez, A. and Varges, S. (eds).
[pdf]
[bib]
[shared task website]
[shared task evaluation (pdf)]
@InProceedings{kelleher:07b,
author = {Kelleher, J.D.},
title = {DIT - Frequency Based Incremental Attribute Selection for GRE.},
booktitle = {In Proceedings of the MT Summit XI Workshop Using Corpora for Natural Language Generation: Language Generation and Machine Translation (UCNLG+MT)},
year = {2007},
pages = {90-92},
}
[hide]
Kelleher, J.D. Attention driven reference resolution in multimodal contexts. Artificial Intelligence Review 25:21-35.
[pdf]
[abstract]
[bib]
In recent years a a number of psycholinguistic experiments have pointed to the interaction between language and vision. In particular, the interaction between visual attention and linguistic reference. In parallel with this, several theories of discourse have attempted to provide an account of the relationship between types of referential expressions on the one hand and the degree of mental activation on the other. Building on both of these traditions, this paper describes an attention based approach to visually situated reference resolution. The framework uses the relationship between referential form and preferred mode of interpretation as a basis for a weighted integration of linguistic and visual attention scores for each entity in the multimodal context. The resulting integrated attention scores are then used to rank the candidate referents during the resolution process, with the candidate scoring the highest selected
as the referent. One advantage of this approach is that the resolution process occurs within the full multimodal context, in so far as the referent is selected from a full list of the objects in the multimodal context. As a result situations where the intended target of the reference is erroneously excluded, due to an individual assumption within the resolution process, are avoided. Moreover, the system can recognise situations where attention cues from different modalities make a reference potentially ambiguous.
[hide]
@article{kelleher:07,
author = {Kelleher, J.D.},
title = {Attention driven reference resolution in multimodal contexts},
journal = {Artificial Intelligence Review},
volume = {25}
pages = {21-35}
year = {2007},
}
[hide]
Costello, F., Kelleher, J.D. and Volk, M. (eds.) Proceedings of the 4th ACL-SIGSEM Workshop on Prepositions at ACL-2007. June 28, Prague, Czech Republic.
[pdf]
[workshop url]
[bib]
@Proceedings{costelloetal:07,
editorr = {Costello, F., Kelleher, J.D. and Volk, M.},
title = {Proceedings of the 4th ACL-SIGSEM Workshop on Prepositions at ACL-2007},
address = {Prague, Czech Republic},
month = {June 28},
year = {2007},
publisher = {The Association of Computational Linguistics},
}
[hide]
Brenner, M., Hawes, N., Kelleher, J.D. and Wyatt, J. (2007) Mediating between qualitative and quantitative representations for task-oriented human robot interaction. In Proceedings of IJCAI-07, Hyderabad, India.
[pdf]
[abstract]
[bib]
In human-robot interaction (HRI) it is essential that the robot interprets and reacts to a human's utterances in a manner that reflects their intended meaning. In this paper we present a collection of novel techniques that allow a robot to interpret and execute spoken commands describing manipulation goals involving qualitative spatial constraints (e.g. "put the red ball near the blue cube"). The resulting implemented system integrates computer vision, potential field models of spatial relationships, and action planning to mediate between the continuous real world, and discrete, qualitative representations used for symbolic reasoning.
[hide]
@InProceedings{brenneretal:07,
author = {Brenner, M., Hawes, N., Kelleher, J.D. and Wyatt, J},
title = {Mediating between qualitative and quantitative representations for task-oriented human robot interaction},
booktitle = {In Proceedings of IJCAI-07},
year = {2007},
location = {Hyderabad, India},
}
[hide]
2006
Kelleher, J.D., van Genabith, J. (2006) A Computational Model of the Referential Semantics of Projective Prepositions. Syntax and Semantics of Prepositions, (ed.) P. Saint-Dizier. Kluwer Publishing.
[pdf]
[abstract]
[bib]
In this paper we present a framework for interpreting locative expressions containing the prepositions in front of and behind. These prepositions have different semantics in the viewer-centred and intrinsic frames of reference (Vandeloise, 1991). We define a model of their semantics in each frame of reference. The basis of these models is a novel parameterized continuum function that creates a 3-D spatial template. In the intrinsic frame of reference the origin used by the continuum function is assumed to be known a priori and object occlusion does not impact on the applicability rating of a point in the spatial template. In the viewer-centred frame the location of the spatial template's origin is dependent on the user's perception of the landmark at the time of the utterance and object occlusion is integrated into the model. Where there is an ambiguity with respect to the intended frame of reference, we define an algorithm for merging the spatial templates from the competing frames of reference, based on psycholinguistic observations in (Carlson-Radvansky, 1997).
[hide]
@InBook{kelleher/vangenabith06,
author = {Kelleher, J.D. and van Genabith, J.},
editor = {Patrick Saint-Dizier},
title = {Syntax and Semantics of Prepositions},
chapter = {A Computational Model of the Referential Semantics of Projective Prepositions},
year = {2006},
}
[hide]
Kelleher, J. and Kruijff, G.J. (2006) Incremental Generation of Spatial Referring Expressions in Situated Dialogue. In Proceedings of COLING-ACL'06. Sydney, Australia. Association of Computational Linguistics.
[pdf]
[abstract]
[bib]
This paper presents an approach to incrementally generating locative expressions. It addresses the issue of combinatorial explosion inherent in the construction of relational context models by: (a) contextually defining the set of objects in the context that may function as a landmark, and (b) sequencing the order in which spatial relations are considered using a cognitively motivated hierarchy of relations, and visual and discourse salience.
[hide]
@InProceedings{kelleher/kruijff:06,
author = {Kelleher, J.D. and Kruijff, G.J.},
title = {Incremental Generation of Spatial Referring Expressions in Situated Dialogue},
booktitle = {In Proceedings of Coling-ACL '06, Sydney Australia},
year = {2006},
organisation = {Association of Computational Linguistics},
}
[hide]
Kelleher, J.D. and Kruijff, G.J. and Costello, F. (2006) Proximity in Context: an empirically grounded computational model of proximity for processing topological spatial expression. In Proceedings of COLING-ACL'06. Sydney, Australia. Association of Computational Linguistics.
[pdf]
[abstract]
[bib]
The paper presents a new model for context-dependent interpretation of linguistic expressions about spatial proximity between objects in a natural scene. The paper discusses novel psycholinguistic experimental data that tests and verifies the model. The model has been implemented, and enables a conversational robot to identify objects in a scene through topological spatial relations (e.g. ''X near Y''). The model can help motivate the choice between topological and projective prepositions.
[hide]
@InProceedings{kelleher/etal:06,
author = {Kelleher, J.D. and Kruijff, G.J. and Costello, F.},
title = {Proximity in Context: an empirically grounded computational model of proximity for processing topological spatial expression},
booktitle = {In Proceedings of Coling-ACL '06, Sydney Australia},
year = {2006},
organisation = {Association of Computational Linguistics},
}
[hide]
Kruijff, G.J. and Kelleher, J. and Hawes, N. (2006) Information Fusion For Visual Refer ence Resolution In Dynamic Situated Dialogue. In Proceedings of Perception and Interactive Technologies (PIT06). Kloster Irsee, Germany. LNCS/LNAI/LNBI Series by Springer Verlag. Eds. Andre, E., Baratoff, G., Dybkjaer, L., Hennecke,
M., Minker, W., Neumann, H. and Weber, M.
[pdf]
[abstract]
[bib]
Human-Robot Interaction (HRI) invariably involves dialogue about objects in the environment in which the agents are situated. The paper focuses on the issue of resolving discourse references to such visual objects. The paper addresses the problem using strategies for intra-modal fusion (identifying that different occurrences concern the same object), and inter-modal fusion, (relating object references across different modalities). Core to these strategies are sensori-motoric coordination, and ontology-based mediation between content in different modalities. The approach has been fully implemented, and is illustrated with several working examples
[hide]
@InProceedings{kruijff/etal:06,
author = {Kruijff, G.J. and Kelleher, J.D. and Hawes, N.},
title = {Information Fusion For Visual Refer ence Resolution In Dynamic Situated Dialogue},
booktitle = {In Proceedings of Perception and Interactive Technologies (PIT’06)},
year = {2006},
editor = {Elisabeth Andre and Laila Dybkjaer and Wolfgang Minker and Heiko Neumann and Michael Weber},
publisher = {Springer Berlin / Heidelberg},
month = {June},
ee = {http://www.springerlink.com/content/e25jn35776r02327/},
series = {Lecture Notes in Computer Science},
volume = {4021},
address = {Kloster Irsee, Germany},
pages = {117 -- 128}
}
[hide]
Costello, F. and Kelleher, J.D. (2006) Spatial Prepositions in Context: The Semantics of near in the Presence of Distractor Objects. In Proceedings of the 3rd ACL-Sigsem Workshop on The Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications, Association for Computational Linguistics. Eds. Arsenijevic, B., Baldwin, T. and Trawinski, B.
[pdf]
[bib]
@InProceedings{costello/kelleher:06,
author = {Costello, F. and Kelleher, J.D.},
title = {Spatial Prepositions in Context: The Semantics of near in the Presence of Distractor Objects},
booktitle = {In Proceedings of the 3rd ACL-Sigsem Workshop on The Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications},
year = {2006},
organisation = {Association of Computational Linguistics},
editor = {Boban Arsenijevic and Tim Baldwin and Beata Trawinski}
}
[hide]
Kruijff, G.J. and Kelleher, J.D. and Berginc, G. and Leonardis, A. (2006) Structural descriptions in human-assisted robot visual learning. In Proceedings of the 1st Annual Conference on Human-Robot Interaction (HRI06). Salt Lake City UT, USA.
[pdf]
[abstract]
[bib]
The paper presents an approach to using structural descriptions, obtained through a human-robot tutoring dialogue, as labels for the visual ob ject models a robot learns. The paper shows how structural descriptions can relate models for different aspects of one and the same ob ject, and how relating descriptions for visual models and discourse referents enables incremental updating of model descriptions through dialogue (either robot- or human-initiated). The approach has been implemented in an integrated architecture for human-assisted robot visual learning.
[hide]
@InProceedings{kruijff/etal:06b,
author = {Kruijff, G.M. and Kelleher, J.D. and Berginc, G. and Leonardis, A.},
title = {Structural descriptions in human-assisted robot visual learning},
booktitle = {in Proceedings of the 1st Annual Conference on Human-Robot Interaction (HRI’06). Salt Lake City, UT USA.},
year = {2006},
}
[hide]
2005
Kelleher, J.D. and Costello, F. and van Genabith, J. Dynamically Updating and Interrelating Representations of Visual and Linguistic Discourse. Artificial Intelligence 167 62-102.
[pdf]
[abstract]
[bib]
The fundamental claim of this paper is that salience - both visual and linguistic - is an important overarching semantic category structuring visually situated discourse. Based on this we argue that computer systems attempting to model the evolving context of a visually situated discourse should integrate models of visual and linguistic salience within their natural language processing (NLP) framework. The paper highlights the importance of dynamically updating and interrelating visual and linguistic discourse context representations. To support our approach, we have developed a real-time, natural language virtual reality (NLVR) system (called LIVE, for Linguistic Interaction with Virtual Environments) that implements an NLP framework based on both visual and linguistic salience. Within this framework saliency information underpins two of the core subtasks of NLP: reference resolution and the generation of referring expressions. We describe the theoretical basis and architecture of the LIVE NLP framework and present extensive evaluation results comparing the system's performance with that of human participants in a number of experiments.
[hide]
@Article{kelleher/etal:05,
author = {Kelleher, J.D. and Costello, F. and van Genabith, J.},
title = {Dynamically Structuring Updating and Interrelating Representations of Visual and Linguistic Discourse},
journal = {Artificial Intelligence},
year = {2005},
volume = {167},
pages = {62-102},
}
[hide]
Kelleher, J.D. (2005) Integrating Visual and Linguistic Salience for Reference Resolution, In Proceedings of the 16th Irish conference on Artificial Intelligence and Cognitive Science (AICS-05). Ed. Creaney, E. Portstewart, Northern Ireland.
Kelleher, J.D. and Kruijff, G.J. (2005) A context-dependent algorithm for generating locative expressions in physically situated environments., In Processdings of the 10th European Workshop on Natural Language Generation. Eds. Mellish, C. and Reiter, E. and Jokinen, K. and Wilcock, G. SIGGEN. Aberdeen, Scotland.
Kelleher, J.D., (2005) Visual Salience and the Other One, In Proceedings of the 6th International Workshop on Multidisciplinary Approaches to Discourse: Salience in Discourse. Eds. Stede, M. and Chiarcos, C. and Grabski, M. and Lagerwerf, L. Pages. 79-88. Publisher: Stichting Neerlandistiek Amsterdam & Nodus Publikationen Munster. Chorin/Berlin, Germany.
Kelleher, J.D. and Kruijff, G.J. (2005) A context-dependent model of proximity in physically situated environments. In Proceedings of the 2nd ACL-Sigsem Workshop on The Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications. Eds. Kordoni, V. and Villavicencio,
A. ACL-Sigsem, Colchester, U.K.
Kelleher, J.D. and Costello, F. (2005) Cognitive Representations of Projective Prepositions. In Proceedings of the 2nd ACL-Sigsem Workshop on The Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications. Eds. Kordoni, V. and Villavicencio, A. ACL-Sigsem, Colchester. U.K.
[pdf]
[abstract]
[bib]
This paper describes a psycholinguistic experiment that investigates the cognitive representations of the projective prepositions in front of and behind. In particular: (1) what is the constituency of
the spatial template (i.e., the prototypical regions of acceptability) associated with these prepositions; (2) what effect does frame of reference ambiguity have a these regions; (3) do these preposi-
tions exhibit a bias towards a particular frame of reference. Our results indicate that: (1) the spatial templates associated with these prepositions correlate with projective spatial templates for above, below, left and right, reported in (Logan and Sadler, 1996); (2) when frame of reference ambiguity occurs these spatial template are modified in a similar manner to that reported in (Carlson-Radvansky and Logan, 1997) for the preposition above; (3) in contrast with the results of previous work (Carlson-Radvansky and Irwin, 1993; Carlson-Radvansky and Logan, 1997) which indicate a preference towards the absolute and viewer-centred frames of reference for the preposition above, the prepositions in front of and behind exhibit a bias towards the landmark's intrinsic frame of reference.
[hide]
@InProceedings{kelleher/costello:05,
author = {Kelleher, J.D. and Costello, F.},
title = {Cognitive Representations of Projective Prepositions},
booktitle = {In Proceedings of the 2nd ACL-Sigsem Workshop on The Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications.},
year = {2005},
organisation = {Association of Computational Linguistics},
editor = {Valia Kordoni and Aline Valencia}
}
[hide]
2004
Kelleher, J.D., van Genabith, J. (2004) Visual Salience and Reference Resolution in Simulated 3-D Environments. Artificial Intelligence Review, Kluwer Publishing. Vol. 21(3).
[pdf]
[bib]
Hawes, N., Kelleher, J.D. (2004) Context Sensitive Word Selection For Single-Tap Text Entry. Accepted for the 2nd European Starting Artificial Intelligence Researchers Symposium at the 16th European Conference on Artificial Intelligence (ECAI'04). Valencia, Spain. IOS Press.
[pdf]
[bib]
Hawes, N., Kelleher, J.D. (2004) Analogy by Alignment: On Structure Mapping and Similarity. Accepted for the 2nd European Starting Artificial Intelligence Researchers Symposium at the 16th European Conference on Artificial Intelligence (ECAI'04). Valencia, Spain. IOS Press.
[pdf]
[bib]
Kelleher, J.D., van Genabith, J. (2004) Exploiting Visual Salience for the Generation of Referring Expressions. In Proceedings of at the 17th International FLAIRS conference. Miami, Florida. AAAI Press.
[pdf]
[bib]
Inverso, S.A., Hawes, N., Kelleher, J.D., Allen, R. and Haase, K. (2004) Think And Spell: Context-Sensitive Predictive Text for an Ambiguous Keyboard Brain-Computer Interface Speller. Biomedizinische Technik, 49(1), pages 53-54.
[pdf]
[bib]
2003
Kelleher, J.D. (2003). A Perceptually Based Computational Framework for the Interpretation of Spatial Language in 3D Simulated Environments, Unpublished Ph.D., Dublin City University, Dublin.
[pdf]
[bib]
Kelleher, J.D., van Genabith, J. (2003) A False Colouring Real Time Visual Saliency Algorithm for Reference Resolution in Simulated 3-D Environments. Proceedings of the 14th Irish Conference on Artifical Intelligence and Cognitive Science (AICS'04). Trinity College Dublin, Ireland.
Kelleher, J.D., van Genabith, J. (2003) A Computational Model of the Referential Semantics of Projective Prepositions. Proceedings of ACL-SIGSEM Workshop: The Linguistic Dimensions of Prepositions and their Use in Computational Linguistics Formalisms and Applications, Toulouse, France.
2002...
Kelleher, J.D., O Nuallain, S., (2001) The SONAS Algorithm for the Interpretation of Spatial Prepositions. Proceedings of Seventh International Colloquium on Cognitive Science (ICCS01). Donostia San Sebastian, Spain.
Kelleher, J.D., Doris, T., et al. (2000). SONAS: Multi-modal, Multi-user Interaction with a Modelled Environment. Spatial Cognition - Foundation and Applications. (ed.) S. Ó Nualláin. Amsterdam/Philadelphia, John Benjamins Publishing.
ONuallain, S., Kelleher, J.D., (1998) Spoken Image meets VRML and Java. Proceeding of MIND 2: Spatial Cognition, Dublin City University, Ireland.