After a few days of reflexion, I finally have decided to share with you the very interesting and quite embarassing experience I have done and that is for me quite revealing of the subjective aspect of listening tests.
As some of you may already have read in my previous posts in the audiophile threads, I used to be a believer of the sound quality enhancements provided by several software and hardware tweaks of the Touch setup. This belief was mainly founded on the many positive listening experiences I made during many endless nights all this last few months.
These are the main improvements done on my system during this period and the perceived benefits of it:
1. Replace stock PSU by S-Booster BOTW linear power supply -> unveiled the music, revealing a lot more details
2. Apply TT2.0 and then TT3.0 -> much cleaner sound, less noisy, more accurate timbers, better soundstage
3. Use Draka FTP cable with shielding removed on a few cm -> less harsch sound with airy and much brighter treble
4. Use digital out through external V-DAC 1 and then 2 -> warmer, richer, rounder sound
5. Use Windows7 instead of NAS -> also warmer, richer sound
6. Tweak priorities and buffersize -> improved soundstage and better tone balance
Some of these improvements were also confirmed by a friend of mine and by my wife.
After all of that, I was so satisfied by the sound produced by my Touch that I decided to build a secundary system around a brand new second Touch. But first, once my second Touch arrived, I took the opportunity to go back to the basics and put my two Touches side by side for a listening test: one will all the mods applied, the other with the stock PSU and config, connected by a low end cat5e patch cord. Both were in sync, with LMS running on win7 and sharing the same DAC by swapping the S-PDIF BJC coax back and forth from one to the other. Those who already read my post on this test already know the outcome: this time I could not notice any significative difference if any.
This was just questioning all the results obtained the all year long during endless white nights and raised a lot of confusion about what to think of all these tweaks.
But there comes the interesting part... something had changed in between that may clarify why I had so contradictory conclusions: the way I was testing, or to be more precise, the mindset I had during the tests!
Let me explain further:
During all the time I spent tweaking my system, the main evaluation criteria I used was: "does it sound different?". I was looking to some relevant details of the music with my new tweaks and then looked if it was better than without tweaks: is it brighter? warmer? more airy? better soundstage?... and I always found extracts of music that confirmed the improvements of the tweak.
But for the side by side testing, questioned by the critiscism of some members of the community about the relevance and the effectiveness of the tweaks, I tried to change of mindset and changed my evaluation method from "does it sound different? is it better?" to "does it sound the same? is it even good?" and guess what...? the differences vanished in the air!!! On every extract that sounded good on the tweaked Touch, I tried to figure out: "how does it sound? what do I hear? is it warm? full of harmonics? airy? with a good soundstage? clean? ..." and then I looked on the untweaked Touch if it sounded the same... and it did!!"
As a bottom line, I still don't know if there is a difference or not but I feel like all these long listening tests were a waste of time, energy and money! Don't understand me wrong: I would love to be able to tweak my Touch to improve the sound quality of my system... but it is so frustrating to see that there is such an expectation bias that it is almost impossible to objectivate differences by simple AB tests.
The reason to open this new thread is that I would like to give the opportunity to other community members to share their testing methods in order to sort out the best practices in critical listening methodology. So feel free to contribute :-)
As some of you may already have read in my previous posts in the audiophile threads, I used to be a believer of the sound quality enhancements provided by several software and hardware tweaks of the Touch setup. This belief was mainly founded on the many positive listening experiences I made during many endless nights all this last few months.
These are the main improvements done on my system during this period and the perceived benefits of it:
1. Replace stock PSU by S-Booster BOTW linear power supply -> unveiled the music, revealing a lot more details
2. Apply TT2.0 and then TT3.0 -> much cleaner sound, less noisy, more accurate timbers, better soundstage
3. Use Draka FTP cable with shielding removed on a few cm -> less harsch sound with airy and much brighter treble
4. Use digital out through external V-DAC 1 and then 2 -> warmer, richer, rounder sound
5. Use Windows7 instead of NAS -> also warmer, richer sound
6. Tweak priorities and buffersize -> improved soundstage and better tone balance
Some of these improvements were also confirmed by a friend of mine and by my wife.
After all of that, I was so satisfied by the sound produced by my Touch that I decided to build a secundary system around a brand new second Touch. But first, once my second Touch arrived, I took the opportunity to go back to the basics and put my two Touches side by side for a listening test: one will all the mods applied, the other with the stock PSU and config, connected by a low end cat5e patch cord. Both were in sync, with LMS running on win7 and sharing the same DAC by swapping the S-PDIF BJC coax back and forth from one to the other. Those who already read my post on this test already know the outcome: this time I could not notice any significative difference if any.
This was just questioning all the results obtained the all year long during endless white nights and raised a lot of confusion about what to think of all these tweaks.
But there comes the interesting part... something had changed in between that may clarify why I had so contradictory conclusions: the way I was testing, or to be more precise, the mindset I had during the tests!
Let me explain further:
During all the time I spent tweaking my system, the main evaluation criteria I used was: "does it sound different?". I was looking to some relevant details of the music with my new tweaks and then looked if it was better than without tweaks: is it brighter? warmer? more airy? better soundstage?... and I always found extracts of music that confirmed the improvements of the tweak.
But for the side by side testing, questioned by the critiscism of some members of the community about the relevance and the effectiveness of the tweaks, I tried to change of mindset and changed my evaluation method from "does it sound different? is it better?" to "does it sound the same? is it even good?" and guess what...? the differences vanished in the air!!! On every extract that sounded good on the tweaked Touch, I tried to figure out: "how does it sound? what do I hear? is it warm? full of harmonics? airy? with a good soundstage? clean? ..." and then I looked on the untweaked Touch if it sounded the same... and it did!!"
As a bottom line, I still don't know if there is a difference or not but I feel like all these long listening tests were a waste of time, energy and money! Don't understand me wrong: I would love to be able to tweak my Touch to improve the sound quality of my system... but it is so frustrating to see that there is such an expectation bias that it is almost impossible to objectivate differences by simple AB tests.
The reason to open this new thread is that I would like to give the opportunity to other community members to share their testing methods in order to sort out the best practices in critical listening methodology. So feel free to contribute :-)
Comment