CM3065 Intelligent Signal Processing Assignment Report: Midterm Exercises on Audio Captcha, Steganography & Speech Recognition, Singapore

Please Note:

• You are permitted to upload your Coursework in the final submission area as many times as you like before the deadline. You will receive a similarity/originality score which represents what the Turnitin system identifies as work similar to another source. The originality score can take over 24 hours to generate, especially at busy times e.g. submission deadline.
• If you upload the wrong version of your Coursework, you are able to upload the correct version of your Coursework via the same submission area. You simply need to click on the ‘submit paper’ button again and submit your new version before the deadline. In doing so, this will delete the previous version which you submitted and your new updated version will replace it. Therefore your Turnitin similarity score should not be affected. If there is a change in your Turnitin similarity score, it will be due to any changes you may have made to your Coursework.
• Please note, when the due date is reached, the version you have submitted last, will be considered as your final submission and it will be the version that is marked.
• Once the due date has passed, it will not be possible for you to upload a different version of your assessment. Therefore, you must ensure you have submitted the correct version of your assessment which you wish to be marked, by the due date.

Your overall total word count should not exceed 2500 words (Weighted at 50% of the final mark for the module)

Task 1.1: Web Application with p5.js

The functionality of the web application should meet the following requirements:

1. The application should include the playback controls and effects controls shown in Figure 1.
2. Internally, the effects must be connected in a chain, as shown in Figure 2.
3. The application should include a Record button that allows the user to start/stop recording the processed signal as a WAV file.
4. The application must display both the spectrum of the original sound and the spectrum of the processed sound.

Task 1.2: Enhanced filter effects

1. Enhance the filter effect by adding a type selector that allows the user to select between a low-pass, high-pass or band-pass filter.
2. Allow the user to select between the live microphone input and the pre-recorded audio file as the audio source for the application.
3. Configure a delay audio effect and add this to the audio chain before the dynamic compressor.

Task 1.3: Report

A written report of approximately 500 words that includes:

• A brief description of the main characteristics of each effect and how they have been programmed in task 1.1.
• A brief analysis of the application discussing how the low-pass filter and the master volume effects affect the sound’s spectrum in task 1.1. This should also be illustrated through screenshots.
• A brief description and analysis of the enhanced filter effects in task 1.2.

Task 2.1: Audio Captcha

The goal of this exercise is to create a new Audio captcha method on a web-based audio application using p5.js and its library p5.speech that processes a pre-recorded sound file, sending the processed audio signal to the computer’s speakers or audio output.

The new Audio captcha method should make it difficult for computer applications to automate but the voice is scrambled enough for humans to understand.

1. Design and develop your own approach for randomly generated Audio Captcha with suitable justification on how filtering options available using p5.js and its library p5.speech are used for processing a pre-recorded sound file, sending the processed audio signal to the computer’s speakers or audio output. See Figure 3 as an example of a typical Audio Captcha on an interactive web page.
2. Justify and discuss your implementation with fragments of code and the final result in your report.
3. Highlight future enhancements to strengthen audio captcha capabilities.

Task 2.2: Audio Visualisation with Meyda

Develop an interactive web-based application for visualising music files. The application must be based on p5.js, p5.speech and the JavaScript audio feature extraction library Meyda.

A local DJ has sent you three sounds (Ex2_sound1.wav, Ex2_sound2.wav and Ex2_sound3.wav) and you have to select Meyda audio features that could help represent these sounds visually in an appropriate manner. For example, if the ‘brightness’ of one of the sounds radically changes over time, to select an audio feature that measures the brightness of this sound could be a good choice from the perspective of producing visual impact.

Complete the following table to list the three Meyda audio features selected for each sound and justify your selections.

You could use the image of Figure 4 as an inspiration. The visual variables could include:

1. Number of rectangles.
2. Rectangle size.
3. Rectangle fill colour.
4. Rectangle border size.
5. Rectangle border colour.
6. Rectangle fill colour opacity.
7. Rectangle border opacity.
8. Rectangle rotation.
9. Background colour.

You have the full freedom to choose which audio features to use and how to map them to the visual variables.

Task 2.3: Voice Controller

Incorporate a voice control system to visualise other music files (i.e., Kalte_Ohren_(_Remix_).mp3 *) using p5.speech, that could recognise voice commands such as:

a. Black, White, Red, Blue, Green: to change the background colour to one of these colours.

b. Square, Triangle, Circle, Pentagon: to change the shape of the generated figures to one of these shapes.

(*) Kalte Ohren (Remix) by Dysfunction_AL (c) copyright 2019 Licensed under a Creative Commons Attribution (3.0) license. http://dig.ccmixter.org/files/destinazione_altrove/59536 Ft: Starfrosch, Kara Square

Task 2.4: Report

A written report of approximately 500 words containing the following:

• A description and justification of the audio features and mapping implemented in Tasks 2.1 & 2.2 from a perspective of visual impact.
• A brief description of the voice controller in task 2.3 with implementation results.

Task 3.1: Audio Analysis

The first task consists of analysing a group of suspicious audio files (Ex3_sound1.wav, Ex3_sound2.wav, Ex3_sound3.wav and Ex3_sound4.wav), determining which one contains a four-number secret code. In this case, the spy has used a more sophisticated method. It seems he has used amplitude modulation to ‘move’ the secret code to an ultrasonic range of frequencies, then mixing this code with that of the suspicious audio files.

To solve this task, you must create an application in Python (exercise 3.1) that should detect which one of the audio files seems to contain suspicious data in an ultrasonic range of frequencies and should be able to play the secret code in an audible range of frequencies.

Task 3.2: Embedding hidden messages

The second task consists of creating a more sophisticated algorithm for embedding hidden messages based on the LSB audio steganography method (exercise 3.2). You will create an application in Python and use the audio file Ex3_sound5.wav to embed the secret message ‘An eye for an eye makes the whole world blind’. The application must include an algorithm that performs the opposite operation (i.e. an algorithm able to extract the hidden message embedded in Ex3_sound5.wav).

You could, for example, to distribute the hidden message through non-consecutive audio samples using a random pattern, use more than one least significant bits to hide the secret data, etc.

Task 3.3: Further investigation of state-of-the-art audio steganographic approaches

The final task consists of writing a brief investigation report to identify a state-of-the-art audio steganographic approach not covered in the module. Provide a technical analysis of the characteristics, implementation, benefits and limitations of the approach. Compare this approach with a Least Significant Bit (LSB) algorithm.

Task 3.4: Report

A written report in approximately 1200 words. This report must include:

• Brief description and results of audio file analysis and embedding in tasks 3.1 and 3.2.
• Short review on findings on state-of-the-art audio steganographic for task 3.3.

Task 4.1: Automatic Speech Recognition (ASR) System

You have to build a prototype of the application (exercise 4) that should meet the following requirements:

1. The application must be written in Python.
2. Your client prefers to host the automatic speech recognition (ASR) software package in the application to avoid slow-downs or interruptions to the system in the event of issues with the internet connection.
3. The client is unsure which speech recognition software package to employ. Use one of the speech recognition systems illustrated in this module and provide your justifications.
4. The application must be capable of language selection and at the very least compatible with the following languages: English, Italian and Spanish (see document Ex4_models.pdf).
5. The airport virtual assistant will be installed in an environment that can be extremely noisy. So, the speech recognition system should be configured to be able to handle this situation. Your client gives you the freedom to implement any solution (for example, to configure in Python a gain/amplification, low pass filter, or some other audio filter to improve the error rate).
6. The company has prepared a set of audio files with which you can evaluate the system. For this evaluation, you will test how well it recognises several phrases in each language. You also have to record and evaluate two short sentences (your_sentence1.wav and your_sentence2.wav). Feel free to prepare your own sentences.

The output of your work should be a table with the following information, where the Word error rate (WER) (*) is the word error rate for the phrase (see Ex4_audio_files.zip):

Please note that in this step of the project, you have to build a prototype, so you have to focus on the functionality of the application rather than on its visual design.

(*) Word error rate (WER) can be computed as:

𝑊𝐸𝑅 = (𝑆 + 𝐷 + 𝐼)/𝑁 × 100

where

• 𝑆 is the number of substitutions,
• 𝐷 is the number of deletions,
• 𝐼 is the number of insertions,
• 𝑁 is the number of words in the sentence

Task 4.2: Investigate on state-of-the-art ASR systems

Identify and critically discuss at least two other recent ASR systems in comparison to the ASR system you used in task 4.1.

Recommend a suitable ASR system that the company should use in the future, or if you do not think any of them are suitable, you should say that. Justify your findings with supportive academic references.

Please note that you are not expected to program other offline ASR systems and compare their results for this task.

Task 4.3: Report

A written report in approximately 800 words. This report must include:

• A brief description of the ASR software package used and how it has been configured.
• A brief analysis of the solution applied to the issue of the noisy environment.
• An analysis of the result of the test based on the set of audio files provided by the client and your own recordings.
• A brief discussion and analysis of other ASR libraries.