ARL Subjective Quality Evaluation Services
ARL staff have more than 20 years of experience in conducting subjective listening tests for
evaluating the performance of speech and audio coding or processing technologies.
We can perform tests on mono, stereo or 5.1 channel content, and presentation can be via either headphones or loudspeakers.
Headphone listening is done in acoustically controlled
IAC sound booth, while
loudspeaker listening is done in a acoustically isolated, double-wall, BS-1116 qualified listening room.
We have created our own software for computer-controlled presention of the stimulus to the subject
and recording of the listener response (see STEP product information page).
ARL can conduct the following types of tests:
- BS.1116:
- MUSHRA:
- Comparative:
- MOS:
ARL can perform all aspects of the listening test, including:
- Design of listening test:
- Preparation of test items:
- Recruiting listeners:
- Creation of STEP scripts:
- Analysis of subjective data:
- Generation of test report:
The following is a list of subjectvive tests in which ARL has
designed, administered, analyzed the data and generated the test report:
| Date | Project |
|---|
| 2006 | | Client: | Major US Telecommunications Equipment Manufacturor | | Project: | Conducted a subjective test to assess the audio quality of a set of six wide band telephone handsets, including two client handsets. Subjective quality was assessed via live, unscripted conversations between two subjects in isolated test rooms for all combination of handset and speakerphone operation. Tasks included designing test, recruiting subjects, administering subjective test, analysis of subjective scores, generation of final report and presentation of report to client. | | Status: | Complete |
|
| 2006 | | Client: | Southwestern US technology company | | Project: | Conducted a subjective test to assess the impact of watermarking technology on motion picture sound tracks. All test items were audio-visual. Tasks included designing test, administering test, analysis of subjective scores and generation of final report. Tests were conducted at a major motion picture audio mixing theatre and subjects were either professional re-mix engineers or experts in theatre sound. | | Status: | Complete |
|
| 2004 | | Client: | MPEG Audio Subgroup | | Project: | Prepare audio test items for a large subjective listening test conducted under the supervision of ISO/IEC WG11 (MPEG) Audio Subgroup. The results of the test were used to determine the core technology for an MPEG work item that would become MPEG Surround. The test consisted of five separate tests, each evaluating different operating modes. Three were 5.1 channel presentations and two were stereo presentations. In each test there were 8 systems under test evaluated at 7 test sites comprising a total of more than 275 listeners. My specific tasks were preparation of test items, creation of STEP scripts for the test , collection of subjective scores from the various test sites, analysis of pooled scores via Excel pivot table and generation of a written test report. | | Status: | Complete |
|
| 2003-2004 | | Client: | 3GPP (Third Generation Partnership Project) | | Project: | Prepare speech and music test items for a very large subjective listening test conducted under the supervision of 3GPP working group TSG-SA WG4. The test comprised 8 low bit rate listening tests and 3 high bit rate listening tests. The item preparation report is 3GPP document S4-040026. | | Status: | Complete |
|
| 2003-2004 | | Client: | 3GPP (Third Generation Partnership Project) | | Project: | Conduct data analysis and generate final report for a very large subjective listening test conducted under the supervision of 3GPP working group TSG-SA WG4. The test comprised 8 low bit rate listening tests at 8 test sites and 3 high bit rate listening tests at 6 test sites. The final test reports are 3GPP documents S4-040099 and S4-040028. | | Status: | Complete |
|
| 1998-2008 | | Client: | MPEG Audio Subgroup | | Project: | In my capacity as Chair of the MPEG Audio Subgroup I have participated in numerous subjective listening test activities. These efforts entail:
- design of listening test
- preparation of test items
- creation of STEP scripts for the test
- collection of subjective scores
- analysis of pooled scores via Excel pivot table
- generation of test report
| | Status: | Ongoing |
|
