Interesting Blind A/B Test Between SONAR PC modules and Mixbus 32C

[McCue]

Mixbus Forum member 'bapu' has conducted a Blind A/B test between SONAR, using it's Pro Channel Console emulator and Mixbus 32C over on the SONAR forums.

When it was revealed which was SONAR and which was Mixbus 32C, the result was pretty much a landslide in SONAR's favour.

Here

[dayvyg]

I didn't read the whole thread but based on the initial test settings I would say the methodology is flawed. If I understand it correctly, he mixed it with the tape and console emulation on in Sonar. Meaning he mixed into this emulation. Then he turned off the emulation and exported it to Mixbus.

All the mix decisions he made with the emulation on in Sonar would be invalid once he turned off the emulation and exported it to mixbus. Essentially, his mix changed the moment he turned off the emulation.

Am I wrong?

d.

[bapu]

I didn't read the whole thread but based on the initial test settings I would say the methodology is flawed. If I understand it correctly, he mixed it with the tape and console emulation on in Sonar. Meaning he mixed into this emulation. Then he turned off the emulation and exported it to Mixbus.

All the mix decisions he made with the emulation on in Sonar would be invalid once he turned off the emulation and exported it to mixbus. Essentially, his mix changed the moment he turned off the emulation.

Am I wrong?

d.

The goal of the A/B was to compare a SONAR 2 track (with console/tape emulation) to a Mixbus 2 Track "Harrison emulation" my attempt was to use the same "raw material" in both.

By turning off the SONAR emulation (once I had a 2 track with the emulation) exporting to Mixbus gave me the "raw" material to import into 32C. I routed each track (Drums, Guitars etc.) to a separate bus in 32C and then exported a two track.

So yeah, I'd say you may be wrong in assuming what I did or (possibly) what the intent was.

Also, I did not "mix" the raw material (i.e. no FX, compression limiting etc) I just kept both systems raw data at unity gain. This was NOT a mix competition.

No biggie though because what I'm finding is that almost everyone has a different idea of how two DAWs should be compared.

I like SONAR, Studio One Pro and Mixbus all for different reasons (and will use them as such). I'm not a evangelist for any DAW and never will be.

BTW I have Reaper, and although I like their approach that a "track" is really anything you'd like it to be, I still have not gotten into it too deeply and I've had all versions from 3 through 5.

[jabney]

Yet another example that most so-called 'blind' comparisons don't work. Why? Because whatever you are comparing has been changed by the comparison itself. Not that you can't compare two things, but it is difficult. You need at least one more 'blind' stage (a 'blind' plus another 'blind'). So the 'audience' has to be ignorant of the real question. Really the 'audience' should be unaware that there is a comparison at all (a 'blind' plus a 'blind' plus a third 'blind'). That usually takes time - i.e. multiple events.

Music and love-making have several things in common; emotion for example. As soon as you keep score, you lose.

[dayvyg]

"I did a mixdown of that project.

Then I turned off all FXs (tape and console) and then did a bus export. "

It sounded like you mixed it first and then disabled the emulation and exported it to MB. Seems you didn't. Fair enough.

Yes, comparing DAWs is a tricky business. A few months ago someone posted the spectre sound studios Mixbus vs Reaper videos on the Reaper forum and suggested that Reaper needs to up its' game. Of course, it met with a lot of criticism. One user even went so far as to do a null test between summing in Reaper and Mixbus and posted it as a reply to said videos.

Personally, I use Reaper and Mixbus. I like them both a lot but for different reasons. I would like to move to Mixbus fully but Reaper has a number of extra features that keep me hanging on. Mixbus is getting there though and their support is top drawer.

d.

[bapu]

Mixbus Forum member 'bapu' has conducted a Blind A/B test between SONAR, using it's Pro Channel Console emulator and Mixbus 32C over on the SONAR forums.

When it was revealed which was SONAR and which was Mixbus 32C, the result was pretty much a landslide in SONAR's favour.

Here

I would say with ONE HUGE EXCEPTION.

The member who put together the SONAR project (and then handed off to me) said:
"What a surprise! I was almost certain that B is Mixbus."

BTW A was Mixbus.

[x42]

Also, I did not "mix" the raw material (i.e. no FX, compression limiting etc) I just kept both systems raw data at unity gain.

What was the research question? What did you expect to learn from that?

[madmaxmiller]

A real comparison would be to have someone importing the same raw wav files into both DAWs (not an export from one of the involved DAWs), do a level balancing and export it the usual way. Then give it to someone who doesn't even know which DAWs were involved and let him listen at precisely the same volume through the same equipment. The person who plays the pieces to the listener doesn't know which is which either. The test must be repeated on a few different days and in random order and the note be made. The person who created the mixes doesn't meet the other two before the tests are over and the result is written and secured.

That comes closest to a real test, I think.

IMHO
MMM

(remember the Mixbus2 vs Mixbus3 sound debate where the null test proved that the results are the same and everyone who found something else had actually a flaw in his methodology - and still some people claimed to "hear" a difference - psychology at its best...)

[Craig]

Mixbus Forum member 'bapu' has conducted a Blind A/B test between SONAR, using it's Pro Channel Console emulator and Mixbus 32C over on the SONAR forums.

When it was revealed which was SONAR and which was Mixbus 32C, the result was pretty much a landslide in SONAR's favour.

Here

Well that settles it then for me, by crackie! I'm deleting mixbus from my system and going back to Sonar!!

Never again will I believe my lying ears !

Craig

[bapu]

Also, I did not "mix" the raw material (i.e. no FX, compression limiting etc) I just kept both systems raw data at unity gain.

What was the research question? What did you expect to learn from that?

Well, the question (experiment) was: is SONAR's CONSOLE/TAPE Emulation better/worse than Mixbus or can you tell the difference.

I was given a SONAR Project that had console emulation enabled on all tracks and buses. No other FXs were used. I exported that as a two track.

Then I turned off all CONSOLE/TAPE EMULATION and exported the SONAR buses (i.e. drums, vocals, bass etc.) and imported those "raw stems" into Mixbus tracks.

Then I routed each MB track into it's own MB bus (turning off track routes to the master bus).

Then I exported that Mixbus Project as a two track. Then I gave the forum members an opportunity to get the two 24/48K stereo files to compare (on their systems).

2 out 7 or 8 people correctly identified (guessed?) which two track was Mixbus. Others were in varying states of guesses and preference and general comments about the (then unknown) SONAR file.

Maybe I'll do it over and this time just take the "true" raw tracks and bus as "identical" to how the tracks were bused in SONAR. As others have pointed out that might be a more accurate test.

And then again maybe I just don't care any more.