TxtDeClip/Remaster: CD de-clipper/ declipping/ restoration
 |
| | | |
New!! DeClip is now called TxtDeClip and part of SeeDeClip Duo
May 2008
| | | | |
| | | | | |
DeClip
contains 16bit integer, 32bit integer and 32bit floating point declipping routines.
- Fast integer maths suit jukebox and fast batch applications, such as declipping on the fly. This technology could be used in top-range CD players.
-
Floating point suits accurate floating point declipping and levelling, useful for preprocessing - for example if you run a radio station you will use the levelling feature to equalise the RMS volume of all tracks. This could be used in high end CD players with for example 20bit DACs.
Digital CD Clipping reconstruction
Put the Lustre back into your CDs with DeClip. This is 44 seconds into track 7: "Over You" in Sheryl Crow's C'mon C'mon album. Some of Sheryls earlier albums are not too bad, but this one is not hi-fi. There is a repeating drum beat that clips all the way through the album, this is about the 5th or 6th noticeable 'splat' sound from a drum and you can see here the original CD waveform and the process of declipping with DeClip.
 Trick5 1 Pre (162.1kB)
|
|
|
Click on each thumbnail to see the transformation. You will notice that the Gnu version gives a very 'polite' declip whereas the Pro version attempts to recreate the dynamics based on the infrmation in the clip - i.e. it prevents any signal from re-entering the clip.
DeClip Pro is now part of the SeeDeClip Duo
package!
Our best ever CD clipping repair tool
The latest declipping R & D put into DeClip have created some brilliant sounding music here at the lab!!
DeClip recent features
- 1 and 2 point clips are now also fixed
-
Declipping of lightly clipped signals significantly improved in quality
-
Declipping of heavily clipped signals significantly improved in quality
-
Resampling
-
Dithering
-
RMS averaging (50ms chunks)
You will be amazed at how good your CDs can now sound, try the demo
and you'll buy the Pro version!
Streaming audio levelling and declipping
Loudness war and CD clipping repair
If you have problematic audio tracks that are clipped, this program will make a best-guess at reconstructing those missing areas. The biggest application is to repair modern CDs, that are deliberately distorted and clipped as a consequence of the 'loudness war', and to incorporate it into your streaming audio system - so you only hear declipped WAV tracks!
For working on CD tracks however we recommend you use the visual side of SeeDeClip Duo
as it is easier to use - especially with the preview feature.
Lost in action - restoring a clipped CD track
Line spaces added for clarity:
> ./txtdeclip -V -g1 15:\ Rock\ My\ Shit\ Uk\ Bonus\ Track.wav out.wav
DeClip ProV2.3-0 written by Graham Wilkinson
Copyright (c) CuteStudio Ltd. 2004-2008, All Rights Reserved.
Reading 'rockshit.wav', fmt: 2ch 44100:16, c1, data: len 3m:52.373333s [10247664], 16bit -> 32bit float.
[0] RMS -9.60dB (0.33/1.00), peak -0.10dB (0.99/1.00)
[1] RMS -10.59dB (0.30/1.00), peak -0.10dB (0.99/1.00)
Scanning [0] max +ve: max 0.9886, -ve: -0.9886
24177 clips found, max gap 5.397ms [238] at 3m:4.543878s [8138385], 0m:3.733492s [164647] lost.
Scanning [1] max +ve: max 0.9886, -ve: -0.9886
14186 clips found, max gap 4.376ms [193] at 1m:2.957098s [2776408], 0m:1.976054s [87144] lost.
Totals
38363 clips (82.54Hz), 0m:5.709546s [251791] lost (max gap 5.397ms [238] at 3m:4.543878s [8138385])
threshold 100/min(1.67Hz) vs 38363 (165.09Hz) (track len 3m:52.373333s [10247664]), verdict -> declip
FIXED: 28482 simple, 4490 copied, 0 inflated, 5908 shifted, 0 left.
[0] RMS -9.37dB (0.34/1.00), peak 5.90dB (1.97/1.00)
[1] RMS -10.45dB (0.30/1.00), peak 5.88dB (1.97/1.00)
Warning: [0] level 2.87dB too high. Warning: [1] level 3.93dB too high. Reducing levels by 3.93dB
[0] scaling by -7.06dB (x0.44) -> RMS -16.43dB (0.15/1.00), peak -1.16dB (0.87/1.00)
[1] scaling by -5.98dB (x0.50) -> RMS -16.43dB (0.15/1.00), peak -0.10dB (0.99/1.00)
Writing 'out.wav', fmt, data, 32bit float -> 16bit.
Damage summary
Since 2.3-0 DeClip Pro also counts and fixes 1 and 2 point clips. Previously these clips were ignored because DeClips level reduction of 6dB rendered them harmless, if still a little inaccurate. However if you are using a CD player or Pro-audio DAC and you may still hear these on the original because the DAC will probably still overshoot and saturate. Therefore it is a valid clip and needs to be counted, and also counts towards any 'noise' on the playback in the clips/second or average Hz 'drone' figure.
This is the summary of the damage:
38363 clips (82.54Hz), 0m:5.709546s [251791] lost (max gap 5.397ms [238] at 3m:4.543878s [8138385])
threshold 100/min(1.67Hz) vs 38363 (165.09Hz) (track len 3m:52.373333s [10247664])
- nearly 6 seconds of music are lost on this one track.
This is the above track, before and after
 Rs 1 Pre (298.2kB)
|
|
|
How it works
The technical stages of restoring an audio track is as follows
- Read into the internal (floating point or integer) format. This needs a certain amount of memory, depending upon the file size and the floating point precision.
If you do not use levelling (-V) DeClip uses a straight 6dB cut of the original signal and internally sticks to 16bit or 32bit integers (depending upon format) and is therefore very fast and compact. If you specify levelling then 32bit floating point is used internally.
-
DeClip: a reconstruction of the lost peaks and troughs in the waveform.
-
Expand (de-compress) if requested. Subject to future improvements. Generally this is mathematically improbable.
-
Mastering - changing floating point/internal format back into 8/16/24/32bit integer WAV files.
16bit CD waveforms are a little short of bits - or resolution.
For instance the quietest signal is 1bit - which is -90.31dB (20.log10(1/(1<<15)) but the next highest value is 2 bits - or 84.29dB.
The difference is around 6dB - or twice the volume.
The loudest bits however count for less due to audios logarithmic scale - 1 bit at the loudest levels only has a 0.000265dB difference (20.log10((1<<15)/((1<<15)-1))).
This means that the quality of the remaining signal becomes less as you lower the volume, so if you have the ability to play 24bit audio files, you should use DeClip to create 24bit output files:
txtdeclip -V -C24 original.wav restored.wav
which will create the best sound you will ever hear from a (previously clipped) 16bit wav file.
Declipping method, light clips
Declipping is ad-hoc and heuristic. It uses very little signal processing theory at all. Having said that, I'm happy that the results of my program look right and sound right. As Google have proved with their language tools, sometimes statistical anaysis is more accurate than the scientific theory that may not always take account of it.
For instance, the majority of clipped tracks are clipped more seriously on one channel that the other. In addition most of the clipped information is identical for both channels, so by simply copying the good data from one channel to the clipped areas of the other channel, one can accurately recreate the signal without any FFT or spline based guesswork. So channel copying is usually faster, more accurate and simpler than any other method. The real struggle comes when both channels are clipped at the same time. This only occurs for very poorly mastered tracks, simple discontinuities make fourier tools largely ineffective, so ad-hoc methods and checks are used in DeClip to treat these very effectively.
DeClip Pro
actually arranges the channels into RMS level order prior to declipping, in order that the lightest clipped track is fixed first and the SIMPLE clips are eligible for later COPY or SHIFT operations to fix the worst channel.
SIMPLE (1 or 2 samples)
Clips of 1 and 2 are ignored. They are not picked up by scanning - but sometimes created as by-products of a semi-successful declip.
A future version may treat a 2-point clip, because technically this is still a clip and the analog domain will overshoot (by up to 3dB).
A 1 point clip might be averaged between the point each side in a later version too, but technically this can be classed as probable compression, and is usually the least worry.
SIMPLE clip elimination (1 to 5 samples)
This is the majority case, and processing of these has changed with the Pro version V2.3. Prior to this the points at the edge of the clips were ignored, the latter versions however recognise that these will be limited in value, and allows then to be corrected. This helps make snare drums and other crisp HF sounds become higher fidelity: i.e. more realistic.
The Pro DeClip version also declips 1 and 2 length clips.
| Declip length | Operation
|
|---|
| 1 point clip | Differential based upon previous and next values
P += f1(δ1, δ2)
DeClip Pro
only
|
| 2 point clip | Differential based upon previous and next values
P1 += f2(δ1, δ2)
P2 += f2(δ2, δ1)
DeClip Pro
only
|
| 3 point clip | Differential based upon previous and next values
P1 += f2(δ1, δ2)
P2 += f3(δ1, δ2)
P3 += f2(δ2, δ1)
|
| 4 point clip | Differential based upon previous and next values
P1 += f2(δ1, δ2)
P2+= f4(δ1, δ2)
P3 += f4(δ2, δ1)
P4 += f2(δ2, δ1)
|
| 5 point clip | Differential based upon previous and next values
P1 += f2(δ1, δ2)
P2 += f4(δ1, δ2)
P3 += f5(δ1, δ2)
P4 += f4(δ2, δ1)
P5 += f2(δ2, δ1)
|
In DeClip Free/GNU (as you may notice in the algorithm) the edge points of the clipped region are left where they are - (on the rail).
Using the first clipped point to guess a curve shape will be inaccurate (because it is probably limited), but it's again better than ignoring it because a lot of the time you're at the top of a small sine wave. 4-5 clipped points suggests a frequency below 5kHz - right in the critical midrange.
As these points are almost certainly supposed to be higher than they are, the result is still slightly compressed.
In DeClip Pro
from V2.3 these points are also corrected, and not just for small clips.
By using intelligent guesses based on the previous unclipped sample values quite often the result looks perfect when checked against non-clipped of quieter other channel waveforms. The effect is to give a more dynamic, realistic sound with more power, air and presence in the music.
The sound of a DeClip Pro
declipped waveform is interesting in the degree of extra life found in the music. It's very nice to listen to.
Serious clip restoration - COPY and SHIFT
Merging waveforms occurs by normalizing the slope of the donor waveform and laying it along the clipped section. In some cases this will still results in a compressed edge of the underlying bass note. Reconstruction of the bass note is tricky however - an FFT or convolution is the best way to predict the required shape of a sine wave - but the original might not be a sine wave anyway, and depending upon the clipping an FFT might create a huge miscalculation and the cure might be worse than the disease!
In DeClip (all versions) all merged waveforms ('shifted' or 'copied') are compressed if required to prevent further clipping. This is to a fixed level of 6dB in DeClip Free, because that level shift is set to 6dB. In DeClip Pro
the level is preset at 6dB but you may vary it to prevent or enhance declip compression.
In DeClip Pro
there is auto incursion detection, any merged segment is checked to see it does not incur into the clip area (as we know it should not - by observing it is a clip), and any incursion corrected by applying a sine correction at the peak.
In addition DeClip Pro
also looks at 'shifted' segments being merged, and because phase information is already lost it is flipped over (180degree phase shift) to suit the direction of the clip.
COPY: using mono information in stereo
Look at the other channel. One channel is usually louder than the other, and you lose stereo for a couple of milliseconds but it's a lot better than the clip, and perfect if the information is pretty mono. Usually it is. The information used from the other channel is not used if it is clipped or has been fixed by shifting (see below).
Alternate channel information is merged in from each side of the entire clipping region, so there remains no clipped points even at the edges. If there is no suitable alternative channel info, the process continues into the next stages (below).
SHIFT: hunting for audio fill-ins
Serious clip restoration - hunting for audio fill-ins
The Free version of DeClip does basic fill in of clips that are clipped on both channels, the Pro version is rather more thorough.
|
