I plan to add more project ideas asap so check back often.
Also if you have any ideas of your own that involve music/audio/audio
in games, feel free to contact me.
Title: Musical Symphony Composer
Description: Musical Symphonies have a well-known structure (four
movements, sonata form etc.). The idea behind this project is
to investigate and implement techniques that would automatically
generate a computer composed symphony. This is a huge goal, so
in reality the project would mainly deal with the musical structure
of the piece. The user would provide the system with a motif or
number of motifs (motif is simply a short musical theme) which
would then be used by the system to create one movement by modifying
the motif, passing it around instrument sections etc. and most
importantly, automatically creating the exposition, development
and recapitulation sections of a symphonic movement. (Don't worry
if you don't understand the musical terms, they're easily learned)
Title: Dynamic Audio Mixing System Based on Image Frame Content
Description: A number of applications have used images as a basis
for creating algorithmic music. The idea behind this project is
to use images as a basis for mixing, the end application being
a dynamic mixing system for games. A simple example would be a
player walking along with the music soundtrack playing. If the
graphics engine renders a sunflare as the player looks around,
the mixing system would detect this from the image frame and add
a piccolo or a chimes line to the music mix. This project would
be an initial investigation into the possibility of such a system
and the creation of a prototype to demonstrate findings.
Title: Musical Tuning Systems Visualizer
Description: There are numerous musical tuning systems available
to musicians, equal-temperament being what we are most used to.
In simple terms, musical tuning systems basically determine how
many notes are in a musical scale and what those notes sound like
(what pitches they are)
http://www.users.dircon.co.uk/~oneskull/3.6.04.htm
for a quick introduction to tuning systems. The idea for this
project is to develop an application that will allow the user
to explore tuning systems both being able to hear them, but also
by being able to visualize them on some form of frequency display
to help them understand the difference between systems. I would
also suggest that the application be compatible with Scala http://www.xs4all.nl/~huygensf/scala/
Title: Physical Models Based MIDI System
Description: A lot of MIDI synths use wavetables or various sound
synthesis methods to create their instruments. This can cause
a number of problems such as MIDI renditions sounding different
on different computers and a lack of expressive control over the
instruments. This project involves creating a suite of physically
based synthesis engines that can be controlled by MIDI data with
an aim to make them as expressive as possible.
Title: Spectral Composers Sketchbook
Description: The main focus of this project is to develop a user
interface for an application which Ive been working on. The interface
will not be your standard widget type UI, it will use the piccolo
Zoomable User Interface Java library http://www.cs.umd.edu/hcil/jazz/
The application itself is designed to help spectral composers
graphically construct a musical score from information extracted
from sound samples. Spectral composition is a compositional approach
that concentrates on the sonority of sounds as opposed to the
more traditional melody/chord progression focus of "traditional"
(I use the term loosely) western music http://www.byz.org/~tildy/spectral.html
The main idea is to automatically extract frequency information
from recorded sounds, allow the composer to graphically specify
which frequency components should be played by which instruments,
how they should interact and then automatically generate a musical
score from this information. While the interface is a major part
of this project, there are other issues which will need to be
addressed such as potentially adding scripting functionality to
the application (possibly using Groovy http://groovy.codehaus.org/).
At the moment, the code I have (written in Java) performs all
the analysis and frequency information extraction and also transcribes
this data directly to a musical score (the application uses http://www.algomusic.com/jmsl/).
However some parts will need to be improved and changed to facilitate
the integration of a front end. While the main focus of this project
is the development and integration of a UI, don't be fooled into
thinking its "simply build a user interface". It will not be a
standard UI and as stated there are more issues than just the
interface development part.
Title: Real-Time Integration of Sound Effects and Music
Description: This project involves the design and development
of a music engine that will integrate sounds into its musical
score in real-time. Such an engine could be used in games or interactive
installations to modify a musical score by somehow using environmental
sounds (for example sounds effects in a game) to enhance the music
that is playing. The idea came about while thinking about alternative
approaches to audio and music in a game environment. For example,
imagine a player walking through an environment such as a forest.
Environmental sound effects such as birds singing or leaves rustling
could be somehow incorporated into the background music. A simple
way of achieving this could be to simply use the effect as a sample
in the music, however such a straightforward technique would likely
blur the line between the sound effects and music. Therefore more
subtle ways should be investigated such as using a spectral music
approach whereby the frequencies present in a sound could be integrated
into the score. http://www.byz.org/~tildy/spectral.html
Technical approaches could include building everything from the
ground up (e.g. DirectX) or possibly using a graphical audio programming
environment such as MAX/MSP http://www.cycling74.com/products/maxmsp
or PD http://puredata.info/
Part of this project would also involve an investigation into
game music in general and alternative approaches to the standard
linear score paradigm.
Title: Musically Controlled Facial Animation
Description: The idea behind this project is to develop an application
that displays a 3D facial model whose expression is controlled
by and reflects the "feel" of music currently playing. It is envisioned
that the Half Life 2 Source engine would be used for the facial
modeling. The Source engine http://developer.valvesoftware.com/wiki/Main_Page
is the games engine behind the Half Life 2 game and the developers
provide an extensive list of applications and SDKs that allow
third parties to develop games and applications based on the engine.
The engine includes impressive facial modeling functionality.
Broadly, the two main challenges of the project are; * Using the
Source engine to perform the facial animations and investigating
if its possible to interface with this functionality using an
API. There is an application called FacePoser which allows a developer
to script facial animations, however this project requires realtime
control. If it is not possible to do this using API calls, an
alternative might be to call a number of facial animation scripts
in realtime. * Determination of the facial expression required
from a give music source (it is based on the "feel" of the music;
happy, sad, intense etc.). To do this algorithmically is very
involved, however for the purpose of the project it may suffice
to base the analysis on something simple like the tempo or dynamics
of the music. Another approach could be to forget about the "feel"
of the music and simply control the movements of the animation
using a characteristic of the music (e.g. animate the face in
time with the beat). While the Source engine is suggested, the
project is not tied to this particular technology.
Title: Real-Time Four-Part Harmonisation using Case-Based
Reasoning
Description: This idea is based on a project that was developed
by two students in previous years (the theses are available from
the library, email me for the student names). They investigated
applying case-based reasoning techniques to automatically harmonise
a melody, adding the Alto, Tenor and Bass parts to a Soprano line
(Bach chorales are a good example of this type of harmonisation).
They also developed a prototype application in Java as proof of
concept and the resulting harmonisations were quite good. This
project involves the design and implementation of a system (most
likely a VST based plug-in http://en.wikipedia.org/wiki/Virtual_Studio_Technology
) that would allow a user to play a soprano line which would then
be harmonised in real-time using the case-based reasoning approach.
Essentially a user would use a MIDI keyboard to play the melody
and the system would provide the other three voices. There will
be many new challenges that will need to be addressed due to the
real-time nature of the application. A possible approach would
be to incorporate some predictive technique to calculate the most
likely notes the player will choose next and calculate a set of
potential harmonisations. Other issues such as what resolution
the harmonisation should be (i.e. every quarter, half note etc.)
will need to be addressed.
Title: Modifiable Music Game / Music-Based Puzzle/Action Game
Description: A number of games have integrated the action of
the gameplay and the music playing to greater or lesser degrees.
http://www.luminesgame.com/uk/
is an example of a tetris type puzzle game where the players actions
and the games reactions directly affect the music (check out the
videos for a good example). http://www.sonicteam.com/rez/e/movie_clips/index.html
is a shooter type game which also modifies the soundtrack based
on the players actions. The idea behind this project is to conceptualise,
design and implement some form of game where music plays an important
and direct role. There are many approaches that could be taken
from a puzzle based game ala Lumines, to a virtual world where
the player interacts with objects in some way which have musical
attributes associated with them. An interesting angle for this
project might be to create some form of game where not only does
the music have such a direct role, but also to base the gameplay
design on the domain OF music. An example might be a tetris style
game except instead of trying to match up geometrical shapes,
the aim is to match up objects that make musical sense. So an
object could fall which represents and plays a C chord, graphically
pulsing in time to the audition of the chord. The next object
might "match up" with the C chord musically and if it is dropped
beside the chord, they would disappear from the soundtrack. This
is just a very general idea, there are tons of issues for such
a game. There is a lot of scope for this project so let me know
of any ideas you might have for a game where music plays a central
role.
Title: Sound Synthesis as a Basis for a Music Focused Game
Description: Following on from the Modifiable Music Game project
idea, the focus of this project would be to create an imaginative
game where music plays a direct role. The unique angle however
would be the use of sound synthesis techniques to produce the
soundtrack. In other words, instead of using sound samples and
MIDI as the basis of the music engine, one would use sound synthesis
and processing techniques. For example, in a "standard" based
music game, when the player performs some action, a recorded sample
(drum kit, guitar chord etc.) is played which merges with the
soundtrack. In the synthesis approach you would perhaps modify
some parameter of a sound generating process instead (filter a
sound, add modulation, modify its harmonic spectrum etc.). This
idea has a lot of scope. Perhaps the focus could be on the development
of a fully featured adaptive synthesis music engine that could
be used in a number of games, or you could focus on creating a
specific game as described above. Design wise, while it would
be possible to build it from the ground up, possibly a better
approach would be to use a third party sound synthesis library
e.g. http://www.softsynth.com/jsyn/
