Project: Terra Sonica

Assignment: Demo for a computer-game for the visually impaired

Client: ‘Change the Script’ (Willem van Weelden)

Courseleader: prof. David Garcia (HKU)

Tutor: Ina van der Brug

Online: terrasonica.hku.nl

Mail: terrasonica@student-kmt.hku.nl

January 2000, Hilversum.

 

Table of Contents

1. Introduction

2. Context analyses

                        2.1 context analyses

2.2 where do the end-results fit in?

2.3 which demands followed from that?

2.4 who can use the end-results?

3. Concept

                        3.1 user interface choice

                        3.2 audio-based navigation

                        3.3 game models

                        3.3.1 the waveform model

3.3.2 the space model

3.3.3 the trumpet model

3.3.4 the fluid tree

3.4 concept ideas

                        4. Resources and research

4.1 literature

4.2 internet resources

4.3 contacts

4.4 terra sonica online

4.5 questionnaire

4.6 facts

5. Audio research document

                        5.1 why headphones?

5.2 Direct Sound Xtra

5.3 audio cues and messages

5.3.1 non-speech audio

5.3.2 speech

5.3.3 music

5.4 existing audio-navigation systems

5.5 perception

5.6 links

6. Framework and Story outline

6.1 possible story outline

6.2 framework

 

*Thanks to....

                        *Appendix 1: Questionnaire - results

*Appendix 2: White Paper

1. Introduction

This is the concept-document on the group-project 'Terra Sonica'. It’s part of the end-result of a project within the European Multimedia Master of Arts (EMMA) program at the ‘Hogeschool voor de Kunsten, Utrecht’. (Faculty Art, Media and Technology in Hilversum.)

Our assignment, coming from the multimedia company ‘Change the Script’ (CTS), was to create a working demo for a computergame for the visually impaired (V.I.). CTS wanted us to fulfil this assignment within their ‘conceptlab’, a place where new multimedia projects can be developed and produced.

2.1 Context Analysis (See graphic on separate page)

Analysing the context of the 'Terra Sonica' project basically came down to mapping out all the different ‘entities’ which were involved in the Terra Sonica project, and determining (relevant) relations between these entities. The goal of this was, to get a clear view on what the project was about and with what and whom it was going to deal with. This has helped us to determine which groupmembers were going to focus on what part of the different areas of the project. The context analysis mapped out the necessary internal communication between research and production.

The analysis pointed out the external communication needed towards the client and possible people involved in the external production. Furthermore, the context analysis also gave a view on what was actually going to be produced by the project-team, and what was not.

Finally this analysis could also have been useful for creating an alternative planning, which would not be a time-task based one, as in a "normal" planning, but a process-task based one. However in the current version of the analysis this has not been implemented.

In the current analysis, we can roughly define three groups where the different entities can be placed in.

First there is the project-result; this is the actual product (or in general: what is going to be delivered). In the Terra Sonica project, this product consists of a working game demo, which should be considered as a "proof-of-concept", and a full concept and game design on paper.

Secondly, the project resources that give all the input to the final result.

The third group covers all the different parties that can use the project-result in any way. Those are the clients and the users of the project-result. As the future users also will also beta-test the project-result, they would provide feedback of user-testing back into the field of the project-resources. Since we have not delivered a complete marketable product, we also put a third party producer in the context.

In Terra Sonica, the project resources consist of four main fields, the ‘design field’ playing a central part. This field contains the main resources to create the project-result: game design, sound design and story design. These are fed by the three other main project-resource fields: technology, science & knowledge and co-producers. The field of technology is heavily connected to the field of design, with technology providing possibilities but also putting restrictions on our possibilities. For instance, game technology provides what is possible in game design but also puts restrictions on that. These two fields should be considered the more practical fields; the focus is on creating and production of the project-result.

The field of science & knowledge provides a ‘back-up’ information flow, into the design field. This is the field where research is the most important task of groupmembers to work in. The knowledge coming from the research provides a more theoretical basis for the actual design; in this case covering subjects such as perception studies and statistics on the V.I..

The way people are involved in the field of co-producers is different from the other resource fields. It does not involve us as producers or researchers, but others. In our case you can think of musicians and actors helping to provide content.

The business school, which also resides in this field, has a different place in this analysis. Because business school-students were going to take care of the business plan, we have added it to the context analysis-scheme. Since we don't have the knowledge a business school has, we choose not to produce this part.

Our demo fits into the possibilities and limitations of our target-group; considering computer- and hardware usage, game playing, and navigation possibilities. It also takes into consideration the rare availability of games especially created for the blind. Because our target-group doesn't have much experience with computergames you can't assume a lot of pre-knowledge that you might assume present in the average gameplayer.

-No use of graphics.

-Not creating a game that can only be used with a (force-feedback-) joystick or mouse, because the majority of our target-group doesn't use these items.

-Clear sounds-usage because that's all there is to work with.

Our final product will be used in the by the blind and visually impaired. Secondly, we would like it to be used by seeing people as well. Looking at the target group from a commercial point of view, it's better to focus as well on the V.I. as the non-V.I.. But, because we have worked primarily from a creative point of view, we mostly focused on the blind people. The age of V.I. computer-users ranges between 20-40 years old, so we focus on them. We have tried to make the content of our game attractive to as well the younger and older users in this general group. We have created a demo in English because we are aiming at an international target-group. The target-group in Holland is not big. We've had a lot of interesting contacts from all over the world.

3. Concept

Developing a concept for a game for the blind can in many ways be compared to developing a concept for regular computer games. One "small" difference however, the inability to make use of visual feedback required us to drop our normal frame of reference regarding user interface design.

In human computer interface design, we mostly use three out of five of our senses. Sight, hearing and the tactile. To receive feedback from the computer we use our sight, and a fairly small amount of our hearing. The tactile is only used to be able to operate certain input devices such as a keyboard, and only recently to also receive feedback from force-feedback devices. This means the largest amount of information flows between tactile "input" and the visual "output".

As stated before, in Terra Sonica we will not make use of that important information flow, as you don’t need your eyes. This makes it especially useful and entertaining for the V.I., who therefore also are our main target audience. Besides that, the game should be fun for non-V.I. as well. Therefore we decided to include all game-players that understand English. The V.I. are likely to feel better at home in this environment since non-V.I. will have to adapt to the situation where there is no vision.

The most logical option is to use audio feedback. This appeals to both of our target groups (the Braille-line does not, and is not even standardised). A large amount of game players is equipped with a digital soundcard. The use of force-feedback devices such as ffb-joysticks will only provide an enhanced level of interaction but will not be necessary. A keyboard has proven worthy enough as game input device, so we will focus on that. The assumptions that are used in graphical interfaces are not compatible with our interface, because it's not graphical. The interaction with only audio as feedback will be quite different, and it will take a whole new approach in user interface design.

We won't stick to a text-based game, with the text being spoken out, like an interactive soundscape. Instead, Terra Sonica will use advanced audio techniques available to consumers. This will enable the user to experience a dynamic game-environment within the atmosphere of a developing story line. As freedom of choice and movement is considered a high value in modern game playing, we have developed software to build a highly interactive navigation system, based on 3D audio.

We would have liked to perform some tests on ability of people to 'hear in space' and be able to localise where exactly is the sound coming from. There are many factors that can influence our perception, like the choice of sounds, their timbre and their frequency, so we will try to keep those different factors constant throughout the tests.

Our test consists of three separate tests:

1. Sound source and the listener are not moving. We want to learn how people hear static sound placed in space. Thereby is the distance of minor importance.

2. Sound source or the listener is moving in space. This is of course virtual movement. From this experiment we want to test how people perceive moving sound and if can they follow it (the moving sound). The speed of the moving sound (or the user) influences the ability to follow the movement of a sound, but we want to know where the limit lies.

3. Both the sound source and the listener are moving. Our main question is: is the navigation possible in that situation and if so what should the speed limit of both be?

If it turns out that 2D audio isn't suitable for navigation we have more options. We have used 2D audio, but didn't test it extendedly because of lack of time. However, when the actual game will be produced, some further testing should be done.

One of them is to use stereo for navigation. That way we know for shore the user perceives the source correctly (from left or right).

We can combine stereo (if it proves to be too difficult to follow) with some other forms of navigation which exclude movement in space.

In the process of developing a final game-structure several models have been created. Here we’ll present different models.

*The inlayed illustrations have been enclosed in this document on separate pages.

In an abstract way, Terra Sonica looks much like a waveform, like an audio wave. A wave is always travelling linear through time, fluctuating between top and bottom. In Terra Sonica time is represented by a linear story line. Some adventures make use of the tree model in which the story line can head in several directions. The user will only follow one of all possible paths. Multiple story lines often make a story less good, so we decided to keep it linear (with a twitch though). Most adventure games try to resolve this linearity in providing a very strong story with highly interactive puzzles and total freedom of movement to create a sense of freedom.

In Terra Sonica this will be different. The story line will be compelling. The usual assumption, that the player is the good guy fighting the evil forces, is not being made. Each adventure story has a good and an evil side, but we'll let the user choose what to be throughout the whole game. You can see this as the fluctuation in a waveform. According to the players' behaviour and the history of behaviour he or she has shown throughout the game, the story line will revolve around the player, either stimulating or opposing his or her behaviour. Staying neutral will hardly be possible, while good and evil start pulling you to either side of the extremes. As the player moves more towards good or evil, the opposing forces will start pushing the player back and forth again. In the most elegant way this creates a wave-like shape in the abstract model we designed.

At the start of the game the moral choices that have to be made, will have minor consequences. This way the story line is more of a guide to playing the game. As the game progresses, the choices will have bigger consequences and they will have a greater effect on the player as a person, and the environment he or she plays in. The path the player chooses being either good or evil will be recorded during the game and will affect the players’ personality in the way he or she can make choices and the way the environment reacts to him or her. Therefore making these moral choices will be closely connected with navigating in the game, and interaction with the environment. (See figure 1)

The waveform model needed to be refined in order to make it useable for the production phase. Since we had decided that our game should take place in a 2 dimensional space, we integrated the waveform model with the space model. The space model is actually very simple. This is exactly what you see when you are working with the Editor; a visual representation of sound ‘entities’ and objects in a two-dimensional plane.

The colours in this model represent ‘good’ sounds (blue), ‘evil’ sounds (red) and sounds which blend the world together (magenta). This model implies that the division between good and evil is not as strict as in the waveform model. There is total freedom where a sound, either good or evil, can be in space. This freedom can be restricted by rules; e.g. a good sound can never survive in a place where there are many evil sounds.

In combining the space and the waveform model, the fact that the minima and maxima of the wave are either good or evil (as in the waveform model), does not apply any longer. In the new Trumpet model (See figure 3), the space and waveform models are integrated, and the vertical movement of the player in the waveform model is replaced by its movement in space (two dimensions) and time (one dimension). This creates a three-dimensional shape. (See Figure 2)

The trumpet model was sort of an interim model before getting to the final Fluid Tree Model.

The figure below shows a (simplified) trumpet model. The green line is the physical player following through space-time. In the beginning of the game, the size of the players’ universe is still small. Later on in the game, the universe will expand. In this graphic (fig 3) space is being scaled, which is only done for convenience of artist. In a real situation the space would actually grow, and the world also wouldn’t be static. So the freedom (now in movement) will still grow while the story evolves.

For ease of understanding, the illustrations shown in this chapter are drawn in a two-dimensional way, while the actual model is three-dimensional. So these illustrations should all be placed in a three-dimensional perspective. The trumpet gave a good view on how the freedom of movement, and the scale of the game along the passage of time. However, it was still too abstract to implement a story into it.

That was the first concern in continuing refinement of the model; the dynamic environment in which the player could freely move around, giving a twist to a seemingly linear story line and changing the environment around him. This all sounded very beautiful, but it needed more of a practical approach. On the other side of the concept still was a story line in the form of a dramatic epic. It would almost be impossible to write a compelling story that could be adjustable to any kind of environment and player status. When story and dynamic environment are opposing, in terms of good and evil, you will end up with a massive tree structure. We definitely wanted to avoid that. This is where the fluid part comes in. The fluid tree is a structure that starts as a line, then turns into an ordinary tree structure, but as the tree grows, the difference in content of the story lines in parallel gets smaller and gets more of a general character. Then, at a certain point it is possible to link different branches of the story to a next branch that fits on both branch ends. (See figure 4)

In figure 5, we can see a tree of possible user scenarios in the story line. The story is built out of linear parts, connected through ‘events’. Every "Y" shape is an event. This figure doesn’t show the linear parts.

The decisions in Terra Sonica are about choosing between good and evil. These choices will determine the story line. The story will rapidly have to expand into different scenarios. There are also paths that come together again. The colour gradient in the figure represents whether the player follows a good or an evil path. When putting this in the trumpet model, these colours can also be seen as physical locations in the world of Terra Sonica. Note that every event pulls the player into a different physical location, which is a consequence of his choice. Moving to a specific location can be the choice itself, since moving towards good or evil could be considered a choice. Such events are for instance a part of the demo story line.

The event could also lead or force the player into a different location. The loose event on top of the tree is almost totally purple. That means that it is a neutral event. Since its entrance is neutral, it will fit onto any of the four tree ends. The exits of the event are a bit more red and blue. The idea of a lot of such totally purple neutral events, which could combine into a story line, was interesting. (See Figure 5)

This dynamic tree shows a linear story being possible on different locations at the same time. The ‘Y’-shapes can move in the horizontal plane (in two dimensions in the three-dimensional form) and lock onto the position of the player, when he/she is coming out of the previous event.

A story consisting of dynamically linked neutral events does not mean it is boring or doesn’t affect the player. But it doesn’t make one follow a good or an evil path. Neutral events also don’t affect the player in a good or evil way. This ‘faked’ tree however does not mean that the events can be rendered useless. They can still mean the players possible death or keep him or her busy for a while with a puzzle or a fight.

The "Y" shapes seem to force the players’ movement along a predefined path (note the horizontal plane (x-axis) in the figures represents the two-dimensional space plane in the trumpet model). However we have to see these ‘Y’-shapes (space-time tubes in 3D) as having flexible borders. The player can also sometimes pass through borders, thus leaving the event and going his or her own way. It’s also possible to have an upside down ‘Y’ which kind of sucks the players to a certain event in a certain place. It is also possible for any of the three (or more) legs of the ‘Y’ or the whole ‘Y’ itself, to move through space-time. Either to find the player or to go away from it. Leaving the ‘Y’ metaphor, the ‘Trumpet-tube’ fills itself with a dynamic 3D dimensional network structure in space-time. This is providing a story line. (See Figure 6)

When viewed from a two-dimensional top-view on the space plane, events can be seen as ring-shaped areas in space where ‘something is happening’. The ‘rings’ grow and shrink and deform and will eventually fade away as the events come to an end. We could even see them as actual sound sources (speakers) in the Terra Sonica Editor. Since everything in Terra Sonica that is measurable is sound, the sound sources will have to be (part of) an event. For starters, just the hearing of a sound is an event. The more complex an event is, the more suitable it is to provide a rich story line. Speakers that behave in a complex manner were originally on schedule, but a lack of time forced to leave it out.

The either static, dynamic or guided behaviour of sound sources, the interaction with each other and the interaction with the player, creates the fluid tree shape in the trumpet. Some branches will be important story lines with lots valuable information; other will provide a lot of dynamic real-time interaction. Events can migrate through time or have the user move on a different time-speed. We thought of this to give the player a second chance in being at an important event he or she has missed before

Such an open-event model could provide the most linear and simple stories as well as extremely complex dynamic story lines. Our intention was to have the game start with a fairly small movement space within a linear story line. This will give the beginning player a safe feeling and guide him or her into the basics of the game-play. Later on, the story will continue to grow as a tree for the more experienced player. The borders of the tree the player is moving through will get more flexible and have more holes. Sometimes there will be no events at all and the player is free to go.

Other times the player will be found by the ‘trigger’ of an event. The story forces the player into a certain event. As soon as the player has learned how to survive in this new world, the amount of events and storytelling elements increase. This will enable the player to go his or her own way. By following certain parts of the story line, or going through events, the player will get more experienced in having control over him- or herself and then controlling the world. Finally the player could become the ruler over Terra Sonica. The events that make the Legend of Terra Sonica are subtly mapping out the future of the player.

In the now following you can find game-elements we have used in the demo, but also ideas we didn't use, but can be used in the actual game.

The pure basic matter is sound, this is being transformed in an audio-refinery where it’s being made into: building blocks and half-fabricates.

Out of these building blocks, functional objects are being created. Out of the half-fabricates, specialities and ‘food’ are being produced. Specialities exist out of words/phrases and music/rhythm. Words are being put in story-lines, as well story as music combine in the ultimate form of a sound: a poem.

The player has to earn the right to understand the language of others: that way the isolation that blind people might experience in every day life in the real world is included.

- Let the player be the only one to communicate with certain persons.

- Let the player use the basics of the economy to collect credits/energy.

- Let the speed of the movements of the player vary.

- Create areas where the player doesn’t have to do anything, and can just stand still and enjoy the sounds for example.

- Let the player and/or other characters leave marks behind so he/she knows where he/she has/they have been before.

1. Farmers: they communicate solely on sounds, no words. They are helpful, but shy.

2. 'Bad guys': act totally on instincts. Hardly any self-control. No values like we have. They are not understandable until they want to be, or unless you have a translation-equipment. (See: transcorder in chapter 6)

3. 'Good guys': noble people with strong values. They enmirror the good. They use magic and are also very helpful.

4. The Wise: they are very individualistic. Among their group they use telepathic communication. Frustration is the result of the fact that you, as player, are the only one on Terra Sonica they can't figure out.

5. 'Bully-ghost': they try to distract you, try to get you to leave your path.

6. Oracle: is separate from everything and everyone else. Wants peace on Terra Sonica.

1. The all-known

2. Guide

3. Healer

4. Wizard

5. Task-giver

6. Trader/Salesman

7. Soldiers/Police

8. Workers

9. The evil

10. Thieves

11. Spy

1. Keys: certain audio ‘fits’ other audio. For example: with a low sound you can open a high a sounding door. With a rhythm one can open a combination lock.

2. Money; Razu. Pure energy: you can trade with it.

3. Speed meter: it can be switched on or off. It bleeps in a high speed when you're moving fast, and slowly when you move slowly.

4. Playhall: where you can play audio-pong, use force feedback jojo.

5. Disguises.

6. Messengers: send audio messages.

7. Binoculars: with this you can listen in one direction sounds, that are send behind layers of others.

8. Conductor of sound/energy.

9. Phone.

10. Mask of Truth: who is telling the truth and who isn't? Mind-reader.

11. Recorder: you can bring sound along, and collect it, to use it in later communications. Surroundings: certain audio, which is location bounded and on which you can navigate.

12. Scanner: with this you can ‘see’ walls, doors, etc. You can upgrade it so you can start hearing other sounds.

13. Scores: you have to be careful with everything you get, otherwise you have to upgrade it. If you get sick or tired, you start moving more slowly, and the sound-volume will become lower.

14. Wall: around the game-field there's a wall. If you bump into it, you'll stay in a chaos of noises, until you move backwards again.

-Groups of people whose language the player can’t understand. Only by the way it sounds, the player can understand the meaning of their words.

-Groups of people that talk too fast to understand, so the player will need a device to slow it down.

-Groups of people who can decide whether they want the player to understand them, or not.

-There can be marked points in the world where the player can go to immediately, by pressing a certain key. This makes the game easier in complex surroundings.

-You can let the player collect words and/or phrase fragments, that he/she has to combine to create understandable sentences.

-Characters the player meets on his/her way through Terra Sonica, can change personality, or size, for example. So can the player.

-Easter eggs.

-You can create a playfield for the player where he/she can just play around.

-The player can use some forms of drugs, which make him experience the world around him/her very differently.

-The player can encounter elements that others don’t know the meaning of, but he/she does.

-There can be an oracle/spirit who is completely independent of the other characters, who knows everything and can DO everything.

-Multiple sounds combined in one sound sample played together at the same time. One of the sound streams contains important verbal information, the others don’t. The player will have to guess (using the 1,2 or 3 number keys to choose one) which one it is, and if he/she succeeds, will hear the message. Otherwise he/she will miss it and will have a disadvantage.

-If the player is taking a break in the game, there will have to be a sound repeated every 10 seconds for example, so the player knows he/she is still in ‘pause mode’.

-The player can be accompanied by his/her own footsteps, or a different sound, making it clear that he/she is the one moving. The player should be able to turn that sound off.

-Different levels of energy can exist, for example climbing from level 1 to 7. (See chapter 6) Different people might be able to produce different levels of energy, which makes certain people more powerful than others.

-The Wise might be able to turn the energy of different levels into pure energy, which is the best sort of all.

-Blinden en Slechtzienden in Nederland', deel 1 en 2, Jonker-Verwey Instituut, Drs. Melief, Dr. Gorter, October 1998.

-'Sensation and Perception', S. Coren, L.M. Ward, J.T. Enns 1994.

-'Interactive Writer's Handbook' and D.Wimberly, J.Samsel 1996.

-Several websites, shown on our projectsite under ‘Links’. (Also see links in audio research.)

-Audyssey magazine, an online magazine about games for the blind.

-The audyssey mailinglist, on which Terra Sonica and v.i.p. post mails about game-playing and game development.

-Computerclub Escape in Amsterdam, for V.I. computer users.

-Margot Schwengle, working at 'Looerf', rehabilitation center for the V.I..

-‘Change the Script’, our client, which has provided us with their research.

-Peter 't Hof en Tim Culhane, two experienced blind programmers who have offered their participation in thinking about the demo and testing it.

-About forty contacts by e-mail with V.I. people worldwide.

-The Terralist should have become a platform for opinions and information on games for the V.I. people. We got our own mailinglist through school; the Terralist. We tried to start up this list because we received so many e-mails from all over the world. We weren’t able to get the list going. All the external mail contacts provided us with a lot of tips and information on computer-usability for the V.I..

-Our projectsite: terrasonica.hku.nl. The site contains: logfiles, research, concept, and team-members descriptions. The site will not contain a lot of graphics and we have made it as user-friendly to the V.I. as we knew how. Please mail your reactions to us.

Several research questions have been answered by having people filling out our questionnaire which is also in the handout. We have received some answers from dutch VIP's and from international VIP's. We have processed the results in Dutch. (See appendix)

Our target group turned out be international, pretty much worldwide. In Holland there are about 600.000 visually impaired people. Approximately 16.000 people are completely blind. About 2000 of them have a computer.

Since we're not using graphics in our game, the totally blind are our primary target group. Our secondary target group are the visually (less) impaired, and gamers that do have sight. After all the game should be daring and interesting to them, too. The user of the game must speak and understand English and have access to a computer with CD-Rom.

Visually impaired people (worldwide) between the ages of 12 and 40 are the ones that use a computer regularly. Most of them use a PC. In the United States and Canada most people use speech friendly-software instead of a Braille terminal. In Holland a Braille terminal is more common.

In the USA, little over 4.3 million people are heavily visually impaired or blind, that number increases with about 50.000 people a year. Ten to fifteen percent of them know Braille reading. At the moment we're in touch with vip's from Israel, Australia, USA, Canada, and Great Britain. China accounts for almost one fifth of the world's blind population, five million persons. We didn't find one of them on the Internet yet.

The report from Jonker-Verwey features the most recent figures. They used research methods from a research in Great Britain. The results turned out to be highly representative for the Dutch visually impaired population.

In general a majority of the completely blind people are older than forty, and not from the 'computerised' generation as we know it. It is not possible to get exact figures on visually impaired worldwide. The graduation of the handicap varies, so does the age, and so does the use of software and hardware per country.

The choice of using headphones for the game is so obvious that is seems obsolete having to explain it. Here are some pro’s and con’s of both headphone and a surround speaker system.

The obvious disadvantage of a surround speaker system is the fact that the speakers are rarely placed properly by the user. The second disadvantage is that with a speaker system, one can also hear all the other sounds in the room (background noise) which makes navigation based on sound even more difficult.

One might say that using a headphone has a disadvantage that all sounds seem to be coming out of users head and not being generated around it. This effect depends on a HRTF (Head Related Transfer Function) that is being used. When we hear a sound without the effect of the pinna our brain thinks it is coming from our body, from inside. Because HRTF is a model of an ‘average’ ear it is never specific enough for an individual user, so the effect of sound ‘coming from inside’ is always there.

The big advantage of headphones is the fact that the speakers are where you want them to be, at all times the headphone is being used. That way you don’t have to worry about the technological solution for creating virtual space with sound, but concentrate on the creative process.

For our demo we are using Director Sound Xtra. It enables us to use Macromedia Director to make a demo, but still have 2D audio (which is often falsely referred to as 3D audio).

DirectSound Xtra is a Lingo Xtra to enable Director applications to use Microsoft’s DirectSound API (part of the DirectX/Game SDK).

The DirectSound API supports low-latency mixing and uses the PC sound hardware wherever possible (that is, when a DirectSound driver is available for the sound card). When the driver is available, DirectSound provides mixing with 20 ms of latency or less. Also, sounds use memory on the sound card when available, using less of the computer’s RAM.

In addition, DirectSound provides more control over sounds. You can get and set the sound position, volume, panning and frequency. You can also control and manipulate 3D sounds.

The main factors of sound positioning are volume (interaural level difference); arrival offset (interaural time difference) and muffling (filtering of sound by the head). These factors are implemented in DirectSound’s positioning system.

In DirectSound Xtra it is possible to make sound cones. That means that a sound has a position and a direction. Two separate cones can be defined – an inside and an outside cone. Within the inside cone, the volume is at maximum volume level for that sound source. The volume inside the outside cone gradually decays. The angles of the sound-cones can be altered.

An important factor in our perception of position and movement is the Doppler shift. DirectSound Xtra applies Doppler-shift effects to sounds, based on the listener’s velocity in relation to one or more 3D sounds.

So now we have sounds, placed in space, all with certain behaviour. Listeners position and velocity (Doppler-shift calculation for the user), listeners orientation (defined by two vectors – the top and the front vector) and the rolloff factor (the amount of attenuation that is applied to sounds, based on the listener’s distance from the sound source) are the three important factors to make the interaction between the listener and the audio world created around him/her possible.

Different types of audio that can be used in a human-computer interface (the game):

There are three different methods of giving non-speech auditory feedback to the user:

- using earcons,

- using auditory icons,

- sonification or data auralisation.

The first two methods are short auditory messages. The third method is a way of representing various data in a continuous fashion (soundscapes).

There is quite a lot of ongoing research about these methods, which suggests that there are many shades of grey, especially between the first two methods.

Earcons are audio cues constructed out of simple motives as building blocs. Motives are very short melodies used as individual, recognisable entities.

Auditory icons are real world sounds mapped to computer events by analogy with everyday sound-producing events. They are similar to visual icons in that they both rely on an analogy between the everyday world and the model-world of the computer.

Auralisation is a method where data parameters are mapped to parameters of sound (frequency, amplitude etc) forming individual tones or continuous streams of sound that change with variations in the data.

Ideally, the auralisation should present data so that people can perceive structures at a higher level of organisation.

This method has been used to present (scientific) data in experimental conditions. If we where to use it in our game, we should conduct experiments of our own. That, on it’s own could be a 3 month project.

Speech can be recorded (digitalized) or synthetic. There are a number of applications for the blind users that use synthesised speech. The disadvantage of synthetic sound is that it usually needs some time to getting used to, because it is not as understandable as it should be.

We will be using recorded sound only because of the lack of time to explore different synthetic sound generating systems and the methods of use (programming-wise).

Music can be used to set a mood, to create a certain setting (the same way it is being used in movies, with graphics/film). Music can complete the total picture and produce a kind of dramatic truth.

Care must be taken when music is used frequently, because music can be annoying if the same piece is heard repetitively.

Our project is one of the firsts in its kind. That sounds very exciting, but it also has a downside. The question of design – how to design sounds for such an environment, is a challenging one. As in music, designers or composers work is based on his/her previous work, but also on what others have been doing before him/her in a specific genre. If there hasn’t been much done, than a designer has to figure out how much research and experimenting can help him/her. Where is the line between the research and the creative process?

This question is not unique for our project, but this is the question I asked myself at a certain point. Here I will try to give my answer to this question.

First, let’s look at the research part of the question. When the project started I wanted to know what has already been done in audio-only navigation. Internet was the first place I looked for answers, but I couldn’t find any simply because nothing like this has been done before. One reason for that is that the technology wasn’t available. By that I mean technology on a commercial level. If you are making a computer game, you also want people to play it. With the arrival of DirectSound it was possible to use more than two sound channels and create an illusion of space. 2D audio is a more or less common feature in the most computer games today. Combined with graphics, which also play the most important part.

There is more than DirectSound. Creative Labs have developed Environmental Audio, which is available on their Sound Blaster Live! Cards (http://developer.soundblaster.com/).

There are also a few systems where the spatial sound is converted to stereo; meaning the user needs but a simple stereo sound card and a headphone to listen to it, no extra software or hardware are needed. Qsound (http://www.qsound.ca/)), Lake DSP ( http://www.lakedsp.com/ ) are some of the companies developing these applications.

As I said before, our project is a challenge, because it is one of the firsts in its kind. This means that there is little available on the subject (and the subject is navigation in a 2D audio world). There have been plans to test the system we are using (Direct Sound Xtra) and it’s possibilities, but we didn't have the time for that.

The answer to the question I asked at the beginning of this paragraph, where is the line between the research and the creative process, seems very clear now (in contradiction to the beginning of the project). The research and the experiments should take place before the start of the production process. With the conclusions and the knowledge acquired from those two the optimal result can be accomplished. Too many rational rules can limit the creative process and vice versa. It is, as so often, a middle road between the two best combinations.

The main factors of sound positioning used in Direct Sound are volume, arrival offset and muffling. At least that is the way they are named in the DirectSound Xtra manual. I will try to explain what those are. Volume is interaural level difference, arrival offset is interaural time difference and muffling is the filtering.

-Interaural level difference – is the difference in the sound level between the two ears.

-Interaural time difference – is the difference in the time of arrival of the same sound in the two ears.

-Filtering – it this case is it the filtering by the head and/or pinna of the user.

There is a lot more to tell about the human perception. The perception of space is very complex. There are many different models of the ‘real thing’ (our perception). Some are better than others. DirectX (Direct Sound Xtra) is not the best there is, but it is commercial and our game too.

Most of these models are based on a HRTF. A HRTF is a mathematical model of the influence the head has on our perception. A HRTF is based on the shadowing effects of the head and on pinna cues.

The pinna is essential for the perception of sound in space, especially the height of a sound source. This is not important because we are working with a 2D sound. The third dimension – height is not of importance. But pinna also plays a role in the filtering of sounds that come from the front or the back.

Every pair of ears is different. Our brain adjusts to the unique pair of pinnae. These differences are so big that every HRTF is an approximation. With this in mind it becomes obvious that either a different approach is needed or the user has to get use to the system that is being used. The user can not expect that the virtual world that is being created is the same as the real world.

A totally different, but important aspect of perception is the ability to process audio information presented. How much can we have? The advantage of the visually presented information is that the user can focus on a certain part is the whole is too much. With audio it’s a different story, because audio has time as an important dimension.

There is a differentiation of information presentation in sound and verbal

information. Sound is in this case everything except spoken text. M. Fernstrom conducted some research on this subject.

(http://www.ul.ie/~cscw/mikael/stroop.html ). This is a new field of research, but I am sure that there will be more available soon, because audio is becoming more and more important in the new media.

Our idea was to create a game where the beginning is a ‘tutorial’ for the user, to get use to the virtual world. The second reason to create a tutorial is that nobody, not even a blind user depends totally on audio to navigate in the real world.

We have placed our game in a fantasy world, so there are also no real world sounds in the game. The hazard of making the whole thing too abstract we provided in a help function using which the user always knows where he/she is at the moment. This can sound too difficult, but it is inevitable, also in games that include sound and vision, because they are all taking place in a virtual world (with its own rules).

5.1 Why Headphones?

5.2 Direct Sound Xtra

http://www.directxtras.com/dsound_home.htm

http://www.tactilepix.com/portfolio/cl_gmat1.html

5.3 Audio Cues and Messages

http://hagar.up.ac.za/catts/learner/generossa/s_o_u_n_d.htm http://www.acm.org/turing/sigs/sigsound/position-papers/stifleman.html

http://www.acm.org/sigs/sigchi/chi95/proceedings/shortppr/csa_bdy.htm

http://www.dgp.toronto.edu/OTP/papers/bill.buxton/audio.html

http://www.acm.org/turing/sigs/sigsound/position-papers/kramer.html

http://www.engr.ucdavis.edu/~das/meera/auditory.html

5.4 Existing Audio-Navigation Systems

http://www.santafe.edu/~icad/ICAD96/proc96/back5.htm

http://www.dgp.toronto.edu/OTP/papers/bill.buxton/audio.html

http://www.lakedsp.com/

5.5 Perception

http://arts.ucsc.edu/ems/music/tech_background/TE-03/teces_03.html

http://www.sasked.gov.sk.ca/docs/physics/u5c32phy.html

http://www.unl.edu./secd/psychoa/experiments

http://www.biols.susx.ac.uk/Home/Chris_Darwin/Perception/Lecture_Notes/Hearing_Index.html

http://oorakel.ipe.nl/plomp/

http://engc.bu.edu/~dizon/prec.html

http://www.rane.com/par-h.htm

http://www.cs.tut.fi/~ypsilon/80545/RoomAcoustics.html

http://ear.Psych.Berkeley.EDU/auditory_lab/spatial.html

http://hyperion.advanced.org/19537/

http://online.anu.edu.au/ITA/ACAT/drw/PPofM/INDEX.html

http://personal-pages.lvc.edu/~fry/snyder/1ch8.html

http://www.speakeasy.org/~draught/spataudio.html

http://www.hitl.washington.edu/publications/hollander/thesis-abstract.html

http://www.ul.ie/~cscw/mikael/stroop.html

All I can hear is a lot of voices talking. Some familiar words, but also strange sounds. I move towards where the volume gets bigger. "Want something?", a voice asks. I remember the 'Razu' I have, and I take one out. Let's see what will happen. Something is being put in front of me; I can feel some kind of soundwave. I decide to walk off. Will that thing I bought just 'stay' with me?

I seem to bump into others, I can hear little 'soundcollisions'. "Hey!! Wait! You forgot something, wait!" The voice sounds very friendly, it stops right in front of me. "You forgot your food, here....." Aha, this is food that sounds logical. "It provides you with energy….Of which you could definitely use some….You sound tired. Why don't you come with me to the resting area, follow."

I do not feel any fatigue. Do I feel anything at all? "Your energy levels sound low", the friendly voice says. The resting area sounds like some kind of chill-out zone. There's relaxed talking in low voices. The friendly voice says: "This is where everyone goes to refuel their energylevels." We are resting, I guess. Soothing sounds slowly drip into my ears.

"My name is Laka………….You don’t seem much of a talker", she laughs. "There must be a way for you to communicate, though. We’re gonna find that out. I have this old friend, he always says he ‘knows it all! Let’s hear what he has to say." Sure thing, I am puzzled by all this………Why is Laka helping me?

I started figuring that I should try and survive here. Maybe I would be able to find out how to get back home. Is that my mission, if there is one?

I can hear Laka starting to move: "Follow me!" We seem to be leaving the resting area behind. A busy place is near by. Some kind of market, I also hear water running……. I try to concentrate on Laka’s sounds, she is quite easy to follow. "Freshly refined NRG!! Only five Razu!", someone yells into my ears. That hurt! Someone is trying to sell his stuff LOUDLY! Refined NRG? The refined NRG has a high sound, it’s almost irritating. What should I do? Buy it? Apparently, I have had food already. The person where I bought the food was probably able to hear I needed it. Was I maybe generating a ‘refuel’ sound?

"We gotta get you a transcorder", Laka says. "That should help you out communicating….Wait here, I’ll try and find a slave that sells them cheap." Where’s Laka? Don’t leave me alone, damn it. I am surrounded by crazy, loud marketpeople. "PSSSSSST!!" That scared me! The sound seems to echo around. My ears try to locate the sound, but it is moving. "Wanna buy a transcorder?? Hey, you can't do without it, and you know it......." Is this the person Laka found? "Come on, I haven't got all day, follow me towards a more silent place..." Now what? Where the hell is Laka? Do I hear danger, maybe..I wouldn't know when I would...... Well, I might as well take the risk and find it out the hard way. I definitely need that transcorder, that's for sure. The seller tries to convince me: "Listen, you: this is a great transcorder, a lot of memory to record ANY sound. And, ofcourse, great replay quality! Also suitable for recording voices, a little mic here........ " The voice keeps circling around me, nervously. "The translator is also working fine. It doesn't feature the latest languages, but: for fifteen Razu this is a GREAT deal!" He turns up his nose and pushes me a: "You need it man......."

For some reason the sleazy voice makes me eager to spend my Razu on this shady figure. I can't hear Laka, maybe she won't come back at all. I start taking out my Razu one by one: One, two, three, four,.......fifteen. "No bargain where you come from huh? If anyone asks: you never heard me...." His nagging sound disappears swiftly, I hardly miss the sound when it's gone, it might have been a hidden sound.

I have wandered off from that cosy market. The sound of a chilling wind strikes me.... That wind is all around me. Too much sounds are irritating, but at this very moment, the calm blowing wind is about the only thing I'm hearing. The sound is threatening instead of soothing. Nothing like a summer breeze..... Okay, try and concentrate, there must be an 'orientation-sound' somewhere..........I turn and I turn....... That was something! Tick, tick......I can hear it far away, the wind breaks the sound. I am getting closer. I hear humming: "Tadom, Lali........and then slower, a ticking sound: -tick-tick.....and faster: tick-tick-tick." What the hell is that?? A deep voice is goggling. It almost sounds like it's waking up. Mumbling: "Shit.....lost again..once more." I move closer. What is this sleepy old creature doing? "There you are.........I have been waiting for you. Laka told me she would bring you, you know. Let's play a game!" A game? Being all mysterious and old and wanting to play a game? Crazy. Where is Laka, this must be a mistake. "Listen to me, you're gonna like this; It's fun. If you win, you get a story, so you get information. That can be pretty valuable for a nitwit like you....hihi...... If I win you give me Razu..... My name is Owa." How ironic: An offer I can't refuse, like I have a choice. "The game......It's like this: Everyone in Terra Sonica plays it all his or her life. It's like a basic skill. You can do lots of things with it. Okay, I am talking too much.... This is a Rythymizer. It will produce a rhythm, we imitate the rhythm, by ticking. The rhythms get more complex in each round. We take turns at imitating that rhythm. The machine will tell if you're doing it right. The first one failing, loses....." We’re there, ticking the rhythms, I am actually pretty good at it. So is the old guy. He probably played it many times, will I win? I NEED to hear that story……………I can hear him cringe when he fails to imitate the rhythm.

"Well stranger….you win. Here’s your story:…….Once upon a time different dimensions and layers of energy were coming together somehow, this must have been a great spectacle, I wasn’t there but that’s what I was told….Anyway….So this weird audio-based universe exist: Terra Sonica. Terra means ‘earth’ and Sonica means ‘sound’. There is no real earth here, but you know how it goes with giving names…….hihi. Energy is the most valuable thing around here. A big part of the energy is being transported from the energyfields to the workfields. Workers or slaves refine the energy there. Energy from different levels is produced there, you can always hear energy! It’s all hearable, you just gotta figure out what sounds how……."

"Anyway…..Like in every country, the creatures that were around first, they split up in groups and started to fight over the energy. So that has left us with the two main groups, the Dosun and the Eonisians. They work very hard to get by, keeping their economy going. There is, however, something nobody can do. It’s like a lack of knowledge, a…eehhh….missing soundvibe. There is a certain kind of sound that nobody can create; the U-tone. Maybe it’s possible, but nobody knows how. This is the prophecy: If a sample of a certain soundwave could be brought over to Terra Sonica, we would be able to reproduce the U-tone. It could actually solve a great deal of the problems that Terra Sonica is having today. The U-tone can provide us with huge energy resources. Dosun en Eonis could call a truce and, eventually, stop fighting. Live together in harmony,…….hihi. Some story, huh? I am not finished yet…"

"Seven wise men are ruling over Terra Sonica. They didn’t succeed in bringing Dosun and Eonis to peace yet. They are researching the possibilities to create the U-tone. Because nobody knows when the prophecy, the Legend will finally happen, they are trying to make it all happen themselve……. But it doesn’t look like they will accomplish anything soon….. Makes you wonder how wise they are….hihi. Finally: the reason that I am telling you this……….there have been some rumours about the Legend, lately….."

My mind wanders off…….., a legend, a prophecy even! Since Owa stopped talking I am able to hear the surrounding sounds again.

6.2 Framework

Why Terra Sonica

Terra (Earth) Sonica (Sound) is a world where a visually impaired person doesn’t have to cope with daily life problems. By using their better-trained ears, they can solve big problems like war, destiny, and saving the universe. The reason that we thought of an environment like Terra Sonica is that we wanted a world that was tailor-made for the blind; a world in which vision does not exist. A blind person wouldn’t even ‘notice’ that he or she is blind.

In this world there should be no things you can see, but only hearable things. Everything in this world should exist out of sound. If something doesn’t produce sound, it does not exist. Silence is not attractive.

With this goal in mind we came up with the story of a world in a different dimension. A dimension without light or mass. A world with no top and no bottom.

As far as we know, this world doesn’t exist. So to write a story we needed to know a lot of things. Why is Terra Sonica like it is? What is the history of Terra Sonica? Who and what is living there? What does the population of Terra Sonica do? In this document you will read about the history, the Wise, economy, sounds, creatures, problems, and the reason why the player is in Terra Sonica.

The dimension of our earth and our galaxy is just one of many. All the dimensions cross each other somewhere. They are all connected. If one of them moves, others can get damaged. Since the beginning of time these dimensions have been connected by a layer of energy. Because there was an intense stress between the dimensions the ‘connecting layer’ started to vibrate and produce sound. Pure, raw sound.

At first there was an abundance of raw sound. This raw sound started to resound, and humm fore millions of years. Out of the resounding sounds, Patterns originated. The Patterns bounced, interacted and changed. New Patterns were born, old Patterns died. Slowly and unnoticed the Patterns started to develop themselves into lifeforms. Simple life forms that started reproducing. The Patterns ate and got eaten by other Patterns.

A wide range of creatures evolved and slowly, very slowly life became intelligent. Life changed the energy levels, the amount of energy decreased. The creatures started robbing each other to get energy. They started to divide themselves into groups and they build thick energywalls around their colonies. The groups of people became more and more intelligent and pretty fast they ruled the whole energylayer they were on.

The raw sound got scarce and, therefore, more important. The battle between the people got more and more intense. Technology got more advanced the weapons and armour became bigger and louder and more powerful.

By this time, a great danger existed: The energylayer Terra Sonica was made out of, could get divided and then the dimensions would float apart. When the war would go on the risks where higher than ever. And the people of Terra Sonica knew it. Dosun and Eonis were at war. A war over the energy in the world. A war about power, a war that could end the world.

An ancient legend seemed to be the only hope in Terra Sonica: Apparently there was someone that could end the war, and bring balance back into Terra Sonica. This person would come from a different world. This person would have different powers.

Most people in Terra Sonica don’t believe this story but seven old wise men know the time is here. They try to get the legend to Terra Sonica. They realise that this is the only thing that can save the them, and the dimensions that are connected to Terra Sonica. They put all their effort into bringing the legend to Terra Sonica. After many years of analysing the legend the wise men calculated the place where the legend would come from. A place in an other part of the universe, an other dimension. Earth! The wise men figured out a way to transport the legend from earth to Terra Sonica by putting up a transport portal. In spite of all their efforts, they can not calculate where the legend will enter Terra Sonica. To make sure they find the legend as soon as possible, they send probes through Terra Sonica. The probe has a mission: find the legend, guide him through Terra Sonica and keep him safe.

In Terra Sonica there are 7 old wise men. They are from an ancient time and have very special powers. Unlike other creatures, the wise men are build out of the highest energy form (E5). This makes them nearly immortal; they can read minds and stretch time. They are the last species of a long extinct race and they are older than the countries and cultures in Terra Sonica. They come from a time where the layer was in perfect harmony. Culture and technology where at incredible levels. They were the few lucky ones that somehow changed their own energy levels to the highest form possible.

Suddenly, a disturbance in the layer was noticed. Evil came to being and started to create conflicts. Evil threatened to take over the world and create chaos and war. Seven council members from different parts of the world came together because they knew of each other that evil had not entered their mind.

They realised the dangers and after many days they came to a the inevitable: They could not bring balance back in their world. The layer would be destroyed. They decide to destroy the world themselves, to save the dimensions from floating apart. They had to destroy everything to bring the layer to its original state and so they did.

They put their forces together and stretched time and distance. When they where finished. The layer was clean. Pure raw sound humming in never ending Patterns. Their family was gone, their friends where gone, the world they used to live in disappeared as if it never existed……….they had killed everything. The never ending humm that kept the dimensions from floating apart generated Patterns and the seven wise men saw life originate all over. They saw species come to being and new intelligent lifeforms evolving. There mission was to keep the balance in this universe and keep the dimensions from floating apart and so they did.

Somehow, though, in this world the balance was not the same like before. Conflicts and war kept coming back. The energy fields were still under great pressure. The wise men seemed to lose. Nobody knew what was happening. One of the wise men was evil. Nobody knew that he was the actual embodiment of evil. He had different plans. He wanted to take over the energylayer and generate a New World. He tried to achieve his goals by manipulating and lying to the other wise men. At the time of the game he has almost achieved his goal. The world is in such great disbalance that it should be cleared again. He found a way to destroy only the parts of Terra Sonica he can’t use. This time, he’ll only get rid of the weak people, in the poor parts of Terra Sonica. He wants to bring the world to its original state and get rid of the six other wise men to. They just see the world going down and they can’t do much about it.

But there is one hope; an ancient legend. A person (the player) from a different world could bring balance back into the layer. Their goal is to get this person here and have him bring back the balance. The evil wise men also needs the player, but he needs it for another goal. He wants to use the player to break the layer in two.

The wise men did not rule the population of Terra Sonica. They gave advice to everyone who needed it. The people from Terra Sonica have different opinions about the wise men. Most people have never even seen one of the wise. They are mysterious, even hated, but also loved. Some people think that the wise are gods and some think they are evil. But everybody knows that the wise are a force not to underestimate

The Terra Sonica world

Terra Sonica is flat. No top and no bottom. Outside the borders of Terra Sonica, there is silence. Some have travelled into the silence but they have found nothing. They travelled throughout nothing and had great trouble coming back. Once a wizard travelled through the silence for two years and when he came back he reported that he didn’t hear anything. He was lucky to have made it back, because many people went crazy after a little while in complete silence.

Terra Sonica consists out of ‘soundpattern’. Each Pattern can be seen as an object. Everything consists out of these Patterns, and therefor everything in Terra Sonica produces sound. Inside Terra Sonica it can be very dangerous. There are Patterns of sound that hunt everything down and if you are not armed it’s likely that you will be killed. There are sounds that alter your frequency, which can weaken. Everyone that is living in Terra Sonica has built walls of energy around their homes.

Outside the protective walls of the cities and villages are big ‘plaines’ where it is almost quiet. It seems like there is almost nothing and then giant Patterns with overwhelming sound pass you by. There are nomads that travel through these planes.

Settling was the easiest near the big energy resources. At these locations the villages and cities were built. New energy generated made there, and raw energy is refined. Out of that, more complex things are produced. Industries evolved and cities were born.

Villages and cities are connected with roads. These roads consist out of points of sound that guide the traveller from point A to point B. These roads sometimes are shielded with energy walls, but most roads are long, unprotected tracks that lead through dangerzones. Travellers have to be skilled and armed to fight off these dangers that they will encounter. They use energy shields to protect themselves.

A safer means of transport is a transport portal. These are gates that can send a traveller from one point of the layer to the other part of the layer immediately.

The different countries in Terra Sonica all have build big energy walls around them for protect from their enemies. These walls cut through the whole world. There are different nations in Terra Sonica.

In the game the most important nations are Dosun and Eonis. Dosun and Eonis are at war. They have big cities and even bigger capitals with high technology and big armies. To fund their war, they exploit everything they can lay their hands on. Most sounds in their country have been used to transform it into what ever they needed for the war.

Next to Dosun and Eonis there are poor countries that live in areas where there is almost no energy. These people work as slaves for the Eonisians or get exploited as workers by the people of Dosun.

And then there are nations with a neutral status. These countries have high technology and are very well protected. They trade with the Eonisian and the Dosun people and make great profit of the war. These countries live in great wealth. They have high technology and incredible walls build around them.

Eonis is an acronym for the word ‘noise’. In Terra Sonica Eonis resembles evil, but like in the real world evil is not always evil and Eonis is not purely bad. Eonis is one of the two great powers in Terra Sonica. It is controlled by one leader and ruled with strict hierarchy.

Dosun is an acronym for the word’ sound’. In Terra Sonica Dosun resembles good, but like good in the real world is Dosun not just purely good.

As said before, everything in Terra Sonica consists out of soundpatterns. These Patterns are the ‘building-blocks’ for everything you want to make. These Patterns also are the core of the Terra Sonica Economics. The Patterns are divided in 5 groups/levels ranging from raw sound to highly refined sound. In the game we categorise these groups with the names E1, E2, E3, E4, E5. These sounds are half-fabrics, from which other things are built.

Not every country in Terra Sonica can reach the same energy levels. For this you need technology and knowledge. If you want to create one unit of E2, you need two units of E1. For one E4 you need four E3’s. This is the general rule but in reality a lot of energy goes lost during the process.

E1 is half the strength as E2, and E2 is half as strong as E3. Nearly all creatures and people in Terra Sonica are made out of E1. This is the most natural form in which sound is generated. E1 energy is also the energy, which is used to feed on and to refuel one's energy levels. E2 Energy is always artificial. It is the first refinement process for sound. It is used to build walls. These walls are good to protect yourself from natural enemies like aggressive sound Patterns, but in war it has little effect. It also is being used to store information with, or as a shield against natural enemies. E2 is also being used very often to create cheap transport vehicles. E2 weapons are only useful for hunting but are too weak for war. Pour armies use E2 weapons.

E3 is a much more advanced level of energy. E3 is very effective as a protection shield against most non-military enemies. It is being used for walls around villages and non-strategic targets. It is a good means to store information in and its also being used for transport vehicles. E3 weapons are also very popular in warfare. It has no problem or what so ever to break E1 and goes pretty easily through E2.

Large quantities of E4 are only allowed to the military. E4 is very effective as a protection wall, although it is very expensive. It is being used on strategic points and as shields in combat. The military loves it. E4 totally blasts itself through an E2 protection shield or wall. Most E3 protection walls are broken down pretty easily. Other uses for E4 are transport vehicles that are very strong and very fast. E4 is also being used to store important information for a very long time. It virtually doesn’t disintegrate. But perhaps one of the most important things is that you can make transport portals out of E4. This makes E4 the most popular energy level in Terra Sonica. Only a few nations have it and are able to make it, therefor its price is very high.

E5 is the purest energy form in Terra Sonica. The creation process is the best-kept secret in the layer. Only one nation in Terra Sonica has the knowledge and technology to create small portions out of it. E5 does not disintegrate. It is immortal and incredibly strong. E5 can be used to build transport portals, time stretching devices and portals to some other dimensions. E5 Shields are unbreakable and weapons are almost unstoppable. E5’s disadvantage is that it takes so much energy to create. Most nations mostly use great quantities of E4 that have the same effect, and that is much cheaper. When E5 is being used, mostly it is for walls around highly strategic targets and for shields that are being used in very dangerous places. As a weapon E5 is almost never used because it is to expensive, but it can be used to scatter an E4 energy wall quickly.

This energy is the core of the economics. Refining the energy, creating different things out of the energy and transporting the goods. Not al countries have the same amount of energy. Labour power and technology also are not equally divided. Eonis and Dosun have large amounts of energy but the almost have not raw sound anymore. They buy or steel energy from other countries. There technology is advanced enough for generating E4 but they can’t make E5.

Other countries still have raw sound and sell this. Some countries have the technology to refine sound cheaper and with less loss or to a higher quality. These countries trade with countries that have other things they might want to have goods like information, walls, transporter, raw energy or even entertainment. Some countries have nothing but labourers and these countries offer to work for the other countries. Often they do get exploited.

War

The war between Dosun and Eonis is a battle for the energy. Energy is life, without energy nothing exists. Energy is power.

Countries protect themselves by building thick energywalls. The enemy will try and destroy these walls by attacking with higher energylevels.

"When the war starts, there is an overwhelming interaction of sounds colliding and hitting each other. Energy that is trying to break its way through energywalls of sound. Troops move through the hole in the wall. The enemy tries to stop the attackers with newly generated walls…. Which can be broken down again, if the enemy is fast enough. Screams fade away ……… energy is sucked away by the enemies."

We would like to thank:

HKU, EMMA:
Ina van der Brug
David Garcia
Joost Burger
Eric Spaans
Willem Jan Renger
Janine Huizinga

Change the Script:
Esther Damen
Willem van Weelden
Bram Van Rooy

The actors:
Anabel da Silva
Suzanne Hoenselaar
Gabe McIntyre
Annemieke Delis
Gabe
Humphry
Jacqueline Schmidt
Guido Stulemeyer

Het Looerf:
Margot Schwengle

Computerclub for the blind 'Escape':    
Henk Kortschot

Everyone that filled out the questionnaire
Everyone that reacted, and mailed us.

And:
Debby Marchena, Peter op 't Hof, Paul Deimann.

*This document has been edited by Neeltje van der Heijden and Jacqueline Schmidt

1.  Geslacht: 65% man, 35%vrouw

2.  Leeftijd: uiteenlopend van 19-64

3.  Hoe is de blindheid ontstaan?

     blind geboren: 45%

     blind geworden: 35%

     slechtziend geboren: 10%

     slechtziend geworden: 10%

4.  Gebruik van een computer: 100% maakt gebruik van een computer

5.  Wat is leuk aan het gebruik van een computer?

- contacten leggen 33.5%

- berichten ontvangen en versturen 55%

- spelletjes 10%

- noodzakelijk hulpmiddel 22.5%

- tekstverwerker 55%

- dingen kunnen die voorheen niet mogelijk waren          10%

- communicatie         22.5%

- Internet 22.5%

6.   Wat is het nut van het gebruik van een computer?

zelf teksten kunnen schrijven en corrigeren:    33.3%

mogelijkheid teksten van anderen te lezen:      10%

raadplegen van informatie:         10%

opslaan van informatie                  10%

7.  Hoe is de toegankelijkheid van een computer?

slecht: 10%

matig: 10%

redelijk goed: 45%

goed: 22.5%

8.  Bent u actief op Internet?

Ja: 22.5%

nee: 77.5%

9.  Wat is uw mening over het Internet?

positief:  55%

negatief:  10%

neutraal:  33%

10.  Wat voor soort computer gebruikt u?

PC’s: 100%

11.  Welk geluidskaart type gebruikt u?

geen geluidskaart         33.3%

geluidskaart die soundblaster emuleert 66.6%

12.  Wat voor soort hoofdtelefoon gebruikt u?

geen:  33.3%

wel:   66.6%

13.  Werkt u met Dos of Windows?

Dos:   45%

Windows:  55%

14.  Welke aanpassingen heeft u bij de computer aan gebracht?

- spraak aanpassing

- braille leesregel

- window-bridge

- braudi-spraak

- groot letterpakket

- extra grote monitor

15.  Welke problemen doen zich voor bij het gebruik van deze aanpassingen?

- Met spraak aanpassing krijg je geen goed overzicht van het beeldscherm. Dit is vooral lastig bij nieuwe programma’s of versies. Braille geeft dan een beter beeld.

- Niet alle iconen zijn weer te geven met spraak en de braille regel

- soms zit windows erg vast door Windowbridge

- braille display kan de tekst op het beeldscherm soms niet volgen/weergeven

16. Zijn de aanpassingen duur in aanschaf?

Ja:  100%

nee: 0%

Om als blinden Dos te kunnen gebruiken liggen de kosten rond de 5000,-

Bij Windows: 15.000,- - 20.000,-

Bij de leesregel: 30.000,-

Braudi-spraak: 6.000,-

17.  Speelt u wel eens computerspellen?

ja:  55%

nee: 45%

18.  Indien ja, welke?

- Verschillende: meeste op het gebied van woordenraden, kaartenspelen, vragen-spellen, maar ook een enkele ‘adventure’ (Dracula, strandjutter)

Indien nee, waarom niet?

- het is lastig spellen te vinden die werken met tekst-naar-spraak programma’s

- te saai

19. Speelt u ook spellen die in eerste instantie ontwikkeld zijn voor zienden?

Nee: 45%

ja:  55% (Lingo, galgje, hussel, rad van fortuin)

20. Wat zou u aan de huidige computerspellen veranderd willen zien?

- gebeurtenissen en boodschappen dienen veel meer automatisch uitgesproken te worden

- gebeurtenissen en boodschappen moeten op vaste plaatsen op het scherm staan

- als je tegen de computer speelt, moet ook aangegeven worden wat die doet

- er moet ruim voldoende tijd zijn om het scherm te bestuderen en te reageren

- ze moeten beter bereikbaar worden. Zo zijn Lingo en Rad van Fortuin heel moeilijk te spelen met spraak, en gaan voor ene leesregel wat snel.

- het grafische zou beter omgezet moeten worden in tekst. Alle details die nodig zijn om het spel te spelen, moeten voor de gebruiker opgemerkt kunnen worden

- in het begin moet er een duidelijke uitleg gegeven worden over het spel

- het geheugen moet niet teveel belast worden

21. Zou u het leuk vinden ‘zienden’ te hulp te staan in een spel?

Ja:        33.3%

Nee:       22.5%

neutraal:  22.5%

22. Hoe navigeert u zich in de wereld om u heen?

- op basis van geluid, met behulp van echo-lokalisatie

- van en duidelijk herkenbaar punt tot een ander duidelijk herkenbaar punt: je hebt een vertrekpunt nodig waar je steeds naar terug moet kunnen  keren en van waaruit je verschillende routes kunt gaan proberen

- als het kan langs duidelijke lijnen die zo recht mogelijk zijn: geluidslijnen

- op het geheugen gebaseerd

- op veronderstellingen

- op gevoel, gehoor en neus

- tikken weerkaatsing, bv. van een dak boven je hoofd

23. Hoe vindt u uw weg in een ruimte die u niet kent?

Een respondent vatte goed samen wat de andere ook omschreven:

-eerst onderzoeken van de directe omgeving van de ingang: de deur. Vervolgens afwachten of er herkenbare geluiden te vinden zijn. Dan kiest men vanuit de ingang bijvoorbeeld een bepaalde richting en onderzoekt daar wat er te vinden is. Als het kan zo dicht mogelijk langs de muur lopen. Is die richting niets, of is die onderzocht, dan keert men terug naar de deur en probeert een andere richting. Het lopen van rondjes probeert men te vermijden omdat dat voor verwarring kan zorgen.

24. In hoeverre bent u in staat iets wat u nooit gezien hebt te visualiseren?

- dit hangt voornamelijk af van hoe goed de beschrijver in staat is beelden in je hoofd te vormen

25. Wat betekent geluid voor u?

- het speelt een heel belangrijke rol. Het kan informatie geven. Kan gebruikt worden voor oriëntatie en navigatie. Het is daarnaast ook een belangrijke bron voor ontspanning. Maar ook juist de afwezigheid van geluid kan heel rustgevend zijn.

26. Wat voor gevoel krijgt u bij de volgende geluids-situaties?

1.  zware, bombastische muziek

- onplezierig gevoel, als het langzaam is een zeer droevig gevoel, als het snel is kan het wat irritant zijn.

2.  licht deuntje

- plezierig, vrolijk, opwekkend, en rustig deuntje kan rustgevend zijn

3.  wind door de bomen

- wisselend: bij zelfstandig de weg vinden kan het heel hinderlijk zijn. Maar in aanwezigheid van anderen, kan het iets moois hebben.

4.  branding

- een plezierig gevoel, ontspannend, roept herinneringen op aan fijne vakanties

5.  veel geluiden tegelijk

- als het een samenhang heeft, kan het interessant zijn om de details er uit te halen, om te identificeren, om de herkomst of de locatie vast te stellen. Als het een kakofonie wordt, kan het echter heel irritant zijn.

6.  helemaal geen geluid

- meestal rustgevend, ontspannend, maar het hangt natuurlijk van de situatie af. In spannende situaties kan het een onbehaaglijk gevoel oproepen. Niet weten wat er gebeurt, geen informatie krijgen.

1. Introduction

This report will give an overview of the software that has been developed in order to create the Terra Sonica Demo.

We used Macromedia Director 7 as an authoring tool to create the software. Not being the perfect solution for creating fairly complex software tools like the Terra Sonica Editor, this might seem a strange choice. However, Director 7 proved to be very suited for Rapid Application Development, because of its relatively simple scripting language and good IDE
(Integrated Development Environment).

In the first stage of the project we were planning on using C/C++ with Microsoft’s DirectSound library and even more advanced 3D audio libraries, but mastering the techniques needed to rapidly develop a game demo with such technologies proved to be too difficult and time consuming for both our programmers.

Looking for alternatives, Director 7 was the best choice (in favor of Visual Basic and Java), since both our programmers had a lot of experience with this tool, and more important, a very nice Xtra (a plug-in for Director), provided an easy way to interface with DirectSound, instead having to master the fairly difficult to use C library.

Of course this limited us in the way we could have used the latest audio technologies, but saved us a huge amount of time. Furthermore, using only DirectSound meant that we could create software for a larger target audience, since almost any PC with a soundcard can run audio software based on DirectSound technologies.

Note: From the beginning of the project we decided not to develop for Apple Macintosh and other OS-es, since there are no cross-platform 3d audio technologies available yet.

From the beginning on, it was clear to the programmers that they would have to take a long trajectory in creating the software to be able to come up with a demo of an audio-based adventure game. It would not have been possible to create only the demo in a kind of "hacking it together" fashion, for the following reasons:

We did not yet know about the possibilities of audio navigation in space.

We knew that we would have to deal with large amounts of data representing an interactive audio-world, meaning high memory and CPU usage.

We needed a translation from audio to a graphical user interface, since we are not blind and rely heavily on visual information.

We knew that our audio designer would have to be able to create the audio worlds herself.

We knew starting to program in an elegant way would come in handy during the last couple of weeks before the deadline, when the "hacking-it-together" phase would inevitably start.

Using Director to reach these goals had advantages and disadvantages. We knew we had to take programming in Lingo to the limit, taking a reverse approach in creating software in Director than normally. Luckily Lingo is kind of easy compared to C++ and much more fault tolerant, but it’s also less basic and straightforward, which meant that we had to use tricks and workarounds to accomplish what would have been much easier, and faster, in C++. One major disadvantage though was the inability to use a Version Control System for managing and merging of the source code created by 2 programmers simultaneously. This is subject to the rather stupid way in which Director saves it’s resources into one single file, making it easy for normal use but clumsy and time consuming for multi-programmer development.

We were very happy to see however that the latest version of Lingo supports a kind of Java style code markup and a fairly ok object-model. Thus we were able to use common object-oriented concepts to create complex software in a way that is similar to Java or C++. We were also glad with the built-in database for managing content (the Cast), the simple way in which File I/O could be handled and the possibility to use vector graphics to create a Graphical User Interface. But still we saw it was absolutely necessary to create a so-called "engine", to be able to achieve what we wanted. The main reasons for this decision were:

Representing an adjustable graphical object for a 3d sound source needed a different type of sprite management than Director’s sprite/score management

Being able to create a world larger than the screen dimensions with an unlimited number of 3d sound sources needed a scrolling and zooming viewport on the world

Having the ability to dynamically create and save the worlds/games meant that we could not use pre-imported media, yet had to come up with a file-format describing the world and a content management system which dynamically creates and removes media from the cast database

The DirectSoundXtra we were going to use does not use the Cast as database, or any other database. So we had to create that one ourselves.

However we did not really come to the point of actually programming the game-interaction system, the design of it proved the necessity of an engine even more.

Initially we thought of three programs in which the engine could be used. First, we wanted to use it to create the TS_Tester, an application meant for testing the capabilities and possibilities of 3D Audio. Actually for 2D Audio, since we had already found out that using the

Z-axis in DirectSound3D without the help of visuals would not be effective. Since a lot of games have already proven that 2 dimensions are enough to create a sense of free movement, we were sure this would be sufficient, if not better for audio based navigation.

In a second phase of the project, an interactive shell would be built around the engine to give the creators of the game the ability to create large interactive 2d audio soundscapes with interactive behaviors. The demands of such an editor (TS_Editor) would depend on the kind of games that had to be built into it, but it was very clear that it should at least have the possibilities the engine had. And finally the game demo, which would become our end product, would also need the engine. In fact, the game demo would become nothing more than a player program for game files generated with the editor, without the GUI of course.

With two not too experienced programmers and only two months of developing time, it proved to be impossible to follow the trajectory of 3 more or less separate programs. Instead the engine, only half completed, was directly transformed into the editor, which would also serve as a platform for the demo game. Testing the quality of 2D Audio was done in a very simple game, which was created in only two days. This game was not suitable for extensive user and perception tests, but proved the quality of the DirectSoundXtra in combination with Director as the audio core of our game to be made. It also proved that navigation in a 2D space without the use of graphics was very well possible. We found out that a really good testing platform would be the editor, so we kind of skipped the release of TS_Tester and continued our work on the engine to make an editor out of it. At this point in time the editor is finished enough to implement the game demo into it, let alone some bugs and bad performance issues in the engine.

Since the engine has more or less merged with the editor in the production process, it’s hard to write technical documentation about the two components separately. And to give the editor a WYCIWYP (What you create is what you play) feature, we decided to integrate the game navigation and interaction into the editor as well. This could fairly easy be held separate in terms of user interaction, since creating worlds/games in the editor is mostly mouse/GUI based, while the game interaction is keyboard/audio based. In the editor these two ways of interaction work parallel, seamless and support eachother. The following part might not be interesting for readers not experienced with software design and might rather continue reading part 5, the editor manual.

In its most optimal form, Terra Sonica would run on an abstract engine suitable for all kinds of dynamic interactive HCI’s, especially for games and game editors. In its first form however, it is highly biased towards the Terra Sonica Game Editor and audio-based navigation. The reusability and expandability of it however, should be considered fairly ok. An overview of the features and planned features should give a fairly good idea of what engine could become.

In the current stage of development all of the features below are functional, but far from optimized. Due to time-shortage these objects are highly dirtied by editor specific elements.

Typical for these features is that they could be functional in any computer game in need of such features as these:

Simultaneous playback of an "unlimited" amount of 2D positioned sounds, using Microsoft DirectSound through DirectXtras Inc. DirectSoundXtra in combination with Macromedia Director 7 and Microsoft DirectX (version 3 or higher), on Microsoft Windows 95/98/NT systems. Playback quality depends on audio hardware and available (soundcard) RAM.

SoundManager and Sound objects for managing files in the WAVE format. Management tasks include loading, unloading, duplicating, positioning and playback functions for WAVE files (static mono 16-bit 44kHz format).

SpriteManager, PolySprite and Sprite objects for managing a limited number (1000) of Director sprite channels to represent graphical objects on screen which are individually built up of more than one sprite. Limitation of number of sprites is subject to Director’s maximum number of sprites. The main goal of this sub-engine is to be able to have an unlimited number of PolySprites/Sprites and to effectively display as many as possible of them with only 1000 available channels. This is possible since the world can be bigger than the screen and to decrease LOD (level of detail) when zooming out (see Viewport)

Viewport object to be able to zoom in on, and scroll over a part of the unlimited 2D space in which worlds can be built out of PolySprites. Handles positioning, scaling and clipping of PolySprites and Sprites. The viewport’s size and location are adjustable.

ActionListener behavior, ActionTarget and Action objects to provide the most sprite independent mouse interaction behavior as possible. It resembles the Java event system, where each registered action target should provide a processAction (Action) handler.

2D Vector math routines

Objects which dynamically create #vectorshape cast members into a specified cast. Currently implemented are: Line (Radius), MultiLine (Path), Circle and Arc

 

These features apply to both engine and editor. In a further stage of development, the objects referred to in this list should be made abstract and migrate to the engine, the original objects becoming descendants of the more abstract objects (object inheritance, if it rings a bell). This would make expanding the functionality of the editor far easier:

FileManager object to read and write (text) files in the TSML (Terra Sonica Markup Language). TSML resembles HTML and VRML, using tags to identify separate chunks of data. It can easily be expanded.

MouseManager and KeyboardManager objects to monitor user input.

Speaker GUI object to graphically represent a "2D SoundCone". See chapter 5 for a full explanation and images.

Path and PathNode GUI objects to represent a movement path for PolySprites and Sprites.

 

In its most optimal form, Terra Sonica’s game editor should be an example of what is possible with a game editor suitable for creating all kinds of games. In its current form however, it’s far from complete and highly biased towards creating Terra Sonica Sound Scapes, keyboard-based audio navigation and audio-game-creation user interface for editing. The biggest lacking part, even for the Terra Sonica Demo is the Processor object, which controls object/object interaction and user/object interaction. Now these tasks have been kind of hacked into the other editor and engine objects.

In the current stage of development all of the features below are functional, but far from optimized and complete. Editor functions are minimized to the absolutely necessary in providing the tools to build and interact with a space built out of positioned and moving sounds.

Creation and editing of 2-dimensional soundscapes, using Speaker objects and motion Paths. These soundscapes are controlled with the SonicWorld object.

Real-time rendering of the 2D soundscape in 3D-audio and real-time rendering of its graphical representation, in both editing and interactive playback mode (between which is no difference actually, due to the difference in user in- and output devices between both modes)

Storing on and retrieving soundscapes from disk. See 4.3

Importing files from everywhere into the specified wave directory, so that all source sounds are located in a known position, useful for editing and distributing soundscapes.

Mouse and keyboard-shortcut controlled graphical user interface with menus, buttons and sliders.

Controller objects, for visually adjusting Speaker Cone’s and location, creating paths for speaker movement and linking player position or speakers to a path. See chapter 5 for a full explanation and images.

 

Maybe some day we will have the opportunity to implement the following features, of which we thought they would be necessary for releasing a true beta version of the full engine, editor and game.

Improved vectorshape and bitmap support

Bug removal of 3d sound duplication bug in DirectSoundXtra

Bug removal of non-omni sound cone (probably own bug)

Sprite engine performance improvement (planned to finish)

Stereo sound playback

Vector collision detection

Joystick support

Network support

Shockwave support

Multi-user server support

GUI builder

Physics engine

AI engine

Karma engine

 

Stereo "Fake 2D" Speaker and Ambient Speaker

Merging different Controllers into one unified Controller object

Multiple object selection and editing (planned to finish)

Multiple sounds per speaker (planned to finish)

Trigger objects

Processor object

Flowchart editor for visual interactive behavior creation, or at least a scripting language

Improved object animation, using keyframing, tweening and Bezier curves

Free object movement using path-finding, following, avoiding and other AI

"Fluid Tree" story-telling engine, based on Karma engine

improved player-game interaction in the form of making choices, picking up sounds, creating and manipulating sounds, having dialogs with game characters, automated movement, audio weapons, audio navigation and interaction tools, etceteras

strategic "god-game" engine

Better quality audio (Creative Labs EAX, Aureal A3D 2.0 (both unsupported in Director))

Force-feedback joystick support (unsupported in Director)

 

This should be considered a brief manual, more to give an idea of what the functionality of the current editor is and how can be used, than to learn the inexperienced user to master the software step-by-step.

For in-depth information about the tools and technologies used, check out:

The screenshot below is from engine149. The final version may look different. In the next sub-chapters the different elements of the interface will be clarified.

Legend:

Viewport

Menu’s

Speaker

Motion Path

Listener

SpeakerController

Target cross for paste action

Main edit menu

Sound database

Play Mode menu

Path Mode menu

Speed Slider

Speaker Info

The viewport provides a top-view on the two dimensional space in which soundscapes can be created. The viewport can be zoomed in and out on and scrolled in order to view or edit a specified part of space. This is achieved by holding down Control (zoom) Shift (scroll), clicking on the background and dragging the mouse. When navigating through space with the Listener (6, see 5.2.6), the viewport will scroll according to the listener’s movement.

5.2.2 Menu’s

The menus are used to edit specific parameters of the soundscape, managing the sound database and storing and retrieving files. See 5.2.9 to 5.2.14 for more information.

5.2.3 Speaker

The Speaker is the main building block for creating spatial soundscapes. Since every sound in space must come from somewhere, it has a point of origin. Compared to the real world, Terra Sonica Spaces are far less complex, since there is no reflection of sound. Calculating reflections of sound waves is very time consuming, and only available on the latest hardware accelerated sound cards. Reflections are not possible to calculate with the DirectSound3D library, which is not state-of-the-art technology anymore. But in the concept of Terra Sonica, this technical limitation is somewhat justified by the fact that there is no hard matter in space to reflect sounds. On the other hand one can ask how the sound is transported through space when there is no matter at all?

Anyway, a sound can be placed on a point of origin in space, represented by the red cross in the middle of the circles. The circles represent the so-called minimum and maximum distance. The minimum distance is the distance from the point of origin where a sound doesn’t get louder when you get nearer to the point of origin. This is a bit strange compared to the real world too, but it’s a trick to workaround a limitation in DirectSound3D, which is that it can not amplify a sound, only attenuate it.

The maximum distance is the point from which a sound doesn’t get softer anymore. We tweaked this parameter a little, to give sound more of a bordered character. To do this, a sound is muted when the Listener is outside of the maximum distance circle. The minimum distance circle can then be adjusted to make the sound either gradually fade in, or make it pop in loud when approaching it.

The lines represent Speaker orientation, and cone settings, which will be clarified in 5.2.7.

To create a Speaker, either select "Create Speaker" from main menu (9.), under item "Create", or use the keyboard shortcut "1". Then click on the background to place the Speaker on its point of origin. To change the Sound a Speaker will play back, select a different wave file from the database menu (10). The Speaker will start playing back the Sound when the Listener enters the Speaker's maximum distance. The playback behavior of the Sound can be one of the following:

"Play once", which will once play the Sound once every time the Listener enters the Speaker cone.

"Looped", which will loop continuously while the Listener is inside the Sound Cone.

"Sequenced", which will become functional as soon as the Processor object has been implemented.

These parameters can be set with a menu (12). Adjusting the Speaker's Cone will be clarified in 5.2.7.

Motion Paths can either be used to make a Speaker or the Listener move along a path. The tie between Speaker and Path is kind of loose, so that a Speaker can be dynamically linked to another Path at runtime. However, such behavior can only be accomplished after the implementation of the Processor object. For now, only the editor of the world can use this feature when creating worlds. It is possible to have more than one Speaker on one single Path, each speaker have their own behavior on the Path. The parameters fro Path behavior are:

Movement speed, which can be set with the GUI slider (13, See 5.2.13)

Looping mode, which can either be "no loop" (one single motion along the Path), "pingpong" (movement back and forth along the Path) or "forward" (continuously loop movement along the Path in one direction.

These parameters can be set with a menu (11). To create a motion path, either selcect "Create Path from the main menu (9), under item "Create", or use the more convenient keyboard shortcut "3". A path consists of PathNodes, in between them a straight line is drawn. By clicking the mouse on the background, PathNodes can be drawn. The right mouse button can be used to stop drawing PathNodes. Important note: In the current version, a major bug in probably the Director software will cause the menu's to disappear when the right mouse button is clicked. This has to be solved, but it's kind of hard since the bug doesn't occur in Authoring Mode, only in the Projector.

Not displayed in the screenshot is the PathNodeController, which is used for creating paths and for adjusting an existing one. When selecting a PathNode a green circle will appear on the PathNode. Dragging it with the mouse will adjust the location of the PathNode. Holding down Shift while dragging the PathNodeController will move the entire Path.

Linking a Speaker to a Path will be explained in 5.2.7.

The Listener is the position where the player's ears are. It's position is always centered in the Viewport. When the Viewport is scrolled, the Listener's position will move with the Viewport. This is very handy when editting a soundscape, so that you can always hear what you are creating. On the other hand, the position of the Viewport is linked to the player's movement. Player movement is controlled with the arrow keys on the keyboard. Combinations of those keys allow 8 types of movement in total. Rotation of the Listener is technically possible and actually provided in the source code, but left out since it proved to be very disorientating.

Movement speed of