Noise removal, and when to stop

You found the noise reduction, dragged it up until the hiss went, and something else left with it. The room has gone. So has a bit of the voice, though you would struggle to say which bit.

Two habits save most recordings from this. Compare the cleaned version against the original rather than against silence, and stop one notch before it sounds clean.

What does noise removal take away?

Noise removal comes in two broad shapes now. The older kind works from a profile: you give it a few seconds of the recording without speech, it measures which pitches the noise occupies and how much of each, from the low rumble at the bottom to the airy hiss at the top, and subtracts that shape from the whole file. The newer, AI-driven kind needs no profile at all, having learned in advance what speech and noise tend to look like, and it separates them on its own. They are built very differently and they hit the same wall.

The trouble is that your voice is not somewhere else. The airy part of a hiss sits exactly where the s, f and th sounds live. A low rumble sits underneath the chest of a voice. Room tone, meaning the quiet continuous sound of a space with nobody in it, occupies very nearly everything. Take the noise away and you take pieces of the voice that were standing in the same spot.

That is not a flaw in one product. It is what separating one sound from another runs into, and everything in this category meets it, however the machine learning on top is described.

Why does the cleanest version sound wrong?

A voice can lose body or texture when a repair is pushed too far. Listen to what makes this recording believable, whether the intended sound is close and dry or full of the room.

A listener may notice an unnatural voice without knowing which process caused it. Compare the result with the original and listen for the fault you are trying to prevent.

What should you be listening to?

Not the noise. The noise is the thing you are trying to stop hearing, which makes it the worst possible thing to monitor.

Most people clean by soloing the gaps: play the pauses, check the hiss has gone, push further if it has not. That comparison has one direction to travel in and it ends at silence every time.

Do it the other way round. Play a sentence, switch the processing off, play the same sentence again, and ask one question. Does this still sound like the person? Not is it cleaner. Does the voice still have a body, do the consonants still finish, is there still air between the words. Judge it on the speech, with the picture running, because that is the only condition anybody else will hear it in.

And stop watching the waveform. It will tell you the noise has gone long before your ears agree, and it cannot show you the thing you are protecting.

How do you know you have gone too far?

Four tells. Any one of them means back it off.

1. The s sounds have picked up a lisp, a click or a faint whistle. 2. Breaths have vanished, so sentences begin out of nowhere. 3. The background swells up between words and ducks away when you speak, which is the processing breathing. 4. The voice sounds like it is behind glass.

Use separate treatments when they address separate problems, and compare the result after each. Two passes can compound damage as well as reduce it.

And there is a reason to stop a notch early. You have been listening to that hiss for as long as you have been working on the file, and your tolerance for it is now zero. You are also the only person who will ever hear the before and the after. Everybody else gets one version with no reference. They will not miss a background you removed most of, and they will notice a voice that has been thinned.

What the automatic setting is optimising for

Automatic cleanup and the newer AI-driven tools are good, and they are good at one thing: finding where the noise lives and how much of it there is. That is measurement, and a machine measures faster and more accurately than a person with headphones and an evening.

What they cannot know is what the recording is for. The newer ones model the voice as well as the noise and can balance the two, but that judgement still sits outside them. Left alone and pushed hard, they trade quality and legibility away for a cleaner measurement, and the result passes every test the tool knows how to run.

Where the tools stop

Here, tools identify and reduce noise; a person decides how far to take the repair. The test is the recording’s purpose: a speaker’s breath may carry the teaching, and the space around a cello may be worth keeping.

The same judgement runs the other way, and that is the half worth taking from this. If a recording has a little steady background behind a voice that is close and clear, the right move is to leave it alone. A quiet room behind a person is what a room sounds like, and it is doing work you would otherwise have to put back.

Where this decision gets made

Ask a fresh listener what they notice, then use that response alongside the brief and your own comparison. One person missing a fault does not settle the decision, but it can help reset ears that have spent too long chasing it.

Sources and evidence

What here is established signal processing and what is our own method.

What is established. Noise reduction trades against the voice: past a point, taking out more noise also takes out wanted speech, a trade-off measured in peer-reviewed speech-enhancement research (Loizou and colleagues, Journal of the Acoustical Society of America). Newer tools push that trade further, but they estimate rather than recover, so it never disappears entirely.

Our method. Judging the cleaned version against the original rather than against silence, and stopping a notch before it sounds clean, is the practice here.

References.

• Kim, G. and Loizou, P. C. Gain-induced speech distortions and the absence of intelligibility benefit with existing noise-reduction algorithms. pmc.ncbi.nlm.nih.gov. Noise suppression introduces amplification/attenuation distortion; conventional NR does not by itself raise intelligibility. Constraining amplification distortion CAN yield gains; does not establish a viewer-retention effect.