Predictive vector quantization of images using a constrained two-dimensional autoregressive predictor

A novel approach to image compression using vector quantization of linear (one-step) prediction errors is presented in this paper. In order to minimize the image reconstruction error, we choose the optimum predictor coefficients (in a least-squares sense) that satisfy the additional constraint that the energy of the impulse response function of the inverse reconstruction filter is bounded by a small constant C. Further, the code vectors are selected such that the reconstruction error is minimized, rather than the quantization noise for the prediction error sequences. Examples demonstrating the excellent quality of the reconstructed images using our approach at bit rates below 0.65 bit/pixel are presented.

PREDICTIVE VECTOR QUANTIZATION OF IMAGES USING A CONSTRAINED TWO-DIMENSIONAL AUTOREGRESSIVE PREDICTOR V. John Mathews, Randall W. Waite and Thao D. Tran Department of Electrical Engineering University of Utah Salt Lake City, Utah 84112 ABSTRACT A novel approach to image compression using vector quantization of linear (one-step) prediction errors is presented in this paper. In order to minimize the image reconstruction error, we choose the optimum predictor coefficients (in a least-squares sense) that satisfy the additional constraint that the energy of the impulse response function of the inverse reconstruction filter is bounded by a small constant c. Further, the code vectors are selected such that the reconstruction error is minimized, rather than the quantization noise for the prediction error sequences. Examples demonstrating the excellent quality of the reconstructed images using our approach at bit rates below 0.65 bit/pixel are presented. I • INTRODUCTION Traditional methods of image compression have been built around scalar predictive and/or transform coding techniques [4, 7). Vector quantization and related techniques are receiving increased attention because of their ability to achieve data rates that are fractions of a bit/sample without affecting the visual quality of the images drastically [1-3, 6, 9). Our paper is concerned with a novel approach to image data Compression USing vector quantization of (one­step) linear prediction error sequences. As the results show, this method performs better than other algorithms of similar complexity available in literature. The basic ideas involved in the Scalar !redictive 'y'ector ~antization (SPVQ) algorithm may be briefly described as follows: The image to be compressed is processed through a simple linear predictor and the resulting prediction error sequence is vector quantized. At the receiver, the vector quantized error sequence is passed through an appropriate reconstruction (inverse) filter to recreate a quantized version of the original image. Comparing our scheme with traditional predictive quantization schemes (see Fig. 1), we find that the major difference is the fact that CH2461-2J87/0000/0243$01.00 © 1987 IEEE 243 our scheme does not guarantee that the input signal to the reconstruction filter is the same as the output signal of the predictor. Because of this, the reconstruction errors will be larger than the quantization errors. Depending on the nature of the predictor (and the corresponding inverse filter), this increase in error can be very large. In order to minimize the effect of this additional noise, our algorithm does the following two things: 1. Instead of using a prediction filter designed to minimize the prediction error power, we will use a "constrained" predictor which is designed to minimize the prediction error power, subject to the constraint that the total energy of the inverse filter (i.e., the sum of squared values of the unit impulse response function of the inverse filter) is bounded by a small constant c. This constraint will ensure that the increase in the reconstruction noise power is not very large. For example, if we assume that the quantization noise sequence is white, it is easy to show that the reconstruction noise power is c times the quantization noise power. Thus the "constrained" predictor design helps reduce the variability of reconstruction error. Our experience is that a choice of c = 2.0 works well for images; i.e., the prediction error sequence is close to the unconstrained prediction error sequence, and at the same ti~e, the reconstruction error is small. 2. The prediction error sequence is quantized using a distortion criterion for the reconstructed signals rather than the error sequence itself. That is, if d(x,y) is our distortion measure and e(n,m), e (n,m), x(n,m) and x (n,m) denote the predicti8n error, quant~zed prediction error, image and reconstructed image sequences, respectively, instead of selecting code vectors that minimize d(e(n,m), e (n,m»), we choose the code vectors that minimi~e d(x,(n,m), x (n,m»). Thus, we strive not to select code ~ectors that produce ~inimum distortion encoding of the error sequence, but to select those that produce minimum reconstruction distortion of the images themselves. The rest of the paper is organized as follows. In the next section, we describe some of the past work done in the area of image compression using predictive and/or vector '111i1/ltl1.iltlon lind then Introduce the Scalar i'r •. ~,llct lve Vector Q\I/lnt izat ion algorithm. We ,,1'If) ,\lac"'''1 our rllil!lOnS for believing that the SI'VI} mf,thod Is 'Hlperlor to the other approaches de!lcrtbed In this section. Section III contains lIm ro~ult!l of lIomo eKperiments with the SPVQ II Iltorlthm. Uo HutMIllrlze our resultg and also .Jl'lCtl'l!I further reflncmentg of the SPVQ algorithm tn Section IV. II. PAST WORK AND TIlE SCALAR PREDICTIVE VECTOR QUANTIZATION ALCOIUTIIH. We wilt hriefly review three different .1I'pro"ch.,.. to dilt." cOl'lpress ton and then describe the Sc.lldr Predictlve Vt'ctor Qtllllltl%ation .11florlthm. Otlll of thelle methoda was propoaed for In\.l-t,,~ Itnd tho other twu for grH!ech signals. ~~tcn~IDn of the concept. Involved In the l~st two appruncha. to d~lll compresalon of Images Is '" t rtll;:ht r orw.lrd. !\"Ir,.>r ,,1\<1 I:ny \1-3) proposed that before veclor '1,,:\ntt1.tnl1, the lm~ge", the s:\11\ple mean of the I"I)(c1q b.'1ol1~lnl1, to ellch vector (block) ought to h,! r~M!)v.HI. nle me'ln re~ ldua I vector 'lll.lnt 17.l1l ton (HRVQ) IltlJ related methods proposed by tht,m h.lYe product codebooks, one subset of the codehook for the s,1mple melln and the other subset f()r tht! re'lldll,ll vect()r. :-ION recent ty, Cuperm,lI\ and Gersho [S J prnp')'Ietl ,\ vector predictive coding scheme for "I1,,!cch slv,l\:ll'l. The veclot" predictive scheme is '!,(,Iclly the '*,1me :ts in Ft~. la, if ... e consider till the ,*IRnall a. vector quantities. Also, the predIctor 15 it vector predictor (I.e., it predicts the next vector based on the present and pllSr. Input vector!!) And the quanth:er is now a vector qURntlzer. The third I!lethod we will discuss Is that introJuceJ by Scht"oeder and Atal \10J for speech signals ~nd 1s known as the "Code E~clted Linear Predlctot"" (CELP). In their ,Ipproach, they use residual codebooks which co~sist of a fatrly large nu~ber of code vectors that are very lon~ (In [IOJ they used a vector length uf 40 ,tnd It codebook size of 1024). Each residual code vector is passed through a synthesis filter H(z) and the output of the synthestg fllter is co~pared with the input 11/i""1 sequence. TI1C rC'lidulll code vector 1electol-i h the one thllt ~tves the Qlninu'll "t"tort ton. ~\)r e,'ch vtctOt". both the index: of t he C'1,!'~ vcctl)r ,lnt! the {Mr.1:n,Hers of the "ynthuls Hlter ::l<ISt I>e tr'\:1snltted. Usln?, the eEL!' ~ch .. ~, Shroeller ,lnd Mill [101 lIere anle to abUt" "(,)tl" IjIl:tl!tl' speech Itt as 10\1 as 4.8 kblt~/~. tr3n'~lsslon rate. The Scalar Predictive Vector Quantization (SPVln al"orithl:1 that lore present next cO'lblnes the good properties of ~ll the above ~ethods and also avoids many of the disadvantages associated \ltth these ~thods. Conceptually, the SPVQ algorlth~ 1s closest to the CELP method. Ho~ever, the SPVQ method can work lIith 244 arbitrarily small vector sizes that also need smaller-size code books. This makes the approach computationally simpler than the CELP method. Scalar Predictive Vector Quantization Algoritlnt. Given an N x M image x(n,m), the SPVQ algorithm consists of the following steps: 1. Partition x(n,m) into smaller, nonoverlap ping blocks of K x L pixels each. Compute the local mean ~ associated with each block. Before the predietor is designed for each of these blocks, the local means must be removed from the image pixels. In our approach, instead of removing the mean values, we will remove a smoothed vet"sion,of the means. Fot" this, define a nell sequence z (n,m) by replacing all x(n,m) In each block by its local mean. Passing this sequence through a smoothing (lo~ pass) filter will yield another sequence for which there is a smooth transition from one block to another. A simple smoothing filter that works well is z(n,m) ~ (1 - y)2 z'(n,m) + y(z(n-1,m) + z(n,m-l)) - y 2 z(n-l, m-l); 0 < y < 1. (I) In Eq. z(n,m) is the output of the filter. 2. Obtain the mean residual sequence y(n,m) by subtracting z(n,m) from x(n,m). Removing z(n,m) instead of the actual local means from the image pixels will eliminate "blocky· reconstructed images. Let laCk, 1);(k, 1)£1!} denote a set of predictor coefficients for the residual sequence in one block. Throu~hout this paper, we will work. with causal predictors. The set of indices (k,1), denoted by 'IT is a finite set of non-negative integer pairs that does not include (0, 0). The transfer function of the predictor is then given by where The transfer function of the inverse (reconstruction) filter corresponding to H(Zl' z2) in Eq. 2 is (2) (4) Let r(n,m) denote the impulse response function of the reconstruction filter. r(n.m) is a causal sequence. The energy e: of r(n,m) is. given by r e: r '" I L n"'O m=O 2 r (n,m). Select the optimum predictor coefficients {a*(k,1)} so that (5) J I I v(n,m) in a e 2 (n,m) (6) given block is minimized, where e(n,m) y(n,m) + L L a(k, t) y(n-k, m-t), (k, t) £11 (7) subject to the additional constraint that e: .. c r (8) where c is a small positive constant. In all the examples and derivations presented in this paper, we worked with a simple, separable predictor with transfer function where I exl, 181 < 1 to ensure stability of the inverse filter. 3. Given the coefficients of the predictor, the codebook and the image sequence in any given block, the sequence can be vector quantized. The vector sizes are usually much s:naller than the block size K x L. In order to vector quantize the sequence, we will pass each code vector through the reconstruction filter and the code vector chosen is that which would produce the minimuM distortion between the image sequence and the output of the reconstruction filter. Since we are using autoregressive predic­tors for the SPVQ algorithm, the reconst ruction filters will have an infinite impulse response (IlR) structure. As a result, the optimal encoding of the residual sequence is very Complex. We will now propose a suboptimal enCoding procedure, that is much simpler computationally. In this approach, image vectors are encoded sequentially so that when each vector is coded, we will assume that the optimal choice of code vectors for all the previous image vectors have been made. As a result, the encoding complexity will only be proportional to the size of the codebook. 4. The indices of the code vectors and the ?redictor parameters along with the block means ::ust be transmitted to the receiver. At the receiver, this information is enough to reconstruct the image by passing the code vectors through the reconstruction filters and adding to the output the smoothed mean values (z(n,m»). Several remarks are in order here. , 1. Even though there is no explicit oeneration of the prediction error sequences, one Can CiJnsider the SPVQ algorithm as a scheme wh~re 245 the prediction errors are first computed and then vector quantized. Vector quantiz1tlon of the prediction errors intriJduces quantization nol~e in these sequences which will in gener.,l he amplified during ima~le reconstructiol,. This in turn implies that the variahility of the quantization noise will he larger for lhe reconstructed images than for the prediction error sequences. Since we woul,\ like to lise relatively small vector and codehook Hize~, it I. very important that the variability of the reconstruction noise is minimlz(!d as much aH possible. The SPVQ algorithm ~chieve'i thlA hy the following two means. a. The code vector,; are cho'len In slIch a way that the reconstruction error rather th;\l1 the quantization error for the prediction error sequence is minimized. b. The design of the "constrained" predictor further guarantees that the variahillty of the reconstruction noise is small. If we .'lssume that the quantization noise for the prediction error sequence" is white, then the reconstruction Daise power Is at most C times that of the prediction error quantization nol~e power. Actually, the reconqtruction noiAe will be smaller than this <lmount dne to step a. Because of this reduced variability, a A~all-~i~e codebook will be able to adequately represent the image pixels involved. 2. The SPVQ algorithm ha~ several conceivable advantages when compared with the three schemes we discussed earlier. As lon~ as the autoregressive model1nl~ is re;lsonably accurate, the prediction error sequence will have a smaller dynamic range than the mean residuals. This indicates that when the same number of code vectors are used, vector quantization of the prediction error seqllences will produce smaller quantization errors than vector quantization of mean residuals as done hy Baker and Gray. Also, the ~RVQ and related algorithms require up to a third of the total number of bits transmitted to convey information about the block means. Tlle ar:l01lnt of 'lide information th<lt must be trans~ltted for the predictor coefficients and block means in the SPVQ approach Is negligible. The vector predictive quantization algorithm [5] of Cuperman and Gerh50 has all the advantages of the scalar predictive codlo?, algorithr:l and also trle~ to take a1vantage of the Inherent superiority of vector 1uantizatliJn over scalar quantization. H'J".tcllcr, in thi<, -;itU,lt lon, one would be predLctin~ the vector b.gc1 on previous vector inputs (eq'Jiv.11ently, the .,c,~l,lr entries of the vector are pre1icte1 by 5~~ple. that are poqsibly as far fro, th~~ a5 the size or a vector). In 'nost practIcal <;itu'ltionq involvin~ images, correlation of qa~?le. (aft~r the mean is extracted) is "luch Qn,111er ,U l.1r;:e distances than ",hen they 'ire :idjacent. Th.!" one can expect to make a better prediction of t~e imafie sequence \lsin\'; sC;llar predict ion tll-m by vector pred ict ion and a9 a result, the overall performance of the SPVQ algorithm should be better than that of the predictive vector '1\1ont1z'1tion scheme. The code-excited linear predictor should perform very well with images. However, as pointed out c:\rl1er, the CELP is a very complex approach to data compression. To make this point more clear, let us consider a specific "tlu.~t1on. To produce very good quality images, the CELP re'1ulreQ fairly large block sizes. "'''!lumln)~ that we URe 32 x 32 hlocks, the method loll II rC'lu1re a code book of approximately 220 code vectorR to encode the residuals using only 1/50 btt per pixel. One can see that the computation.,l complexity involved here is e~tremely large. The SPVQ algorithm makes use of sm,lller-qi;:ed code vectors and codebooks and therefore is a much more simple approach to predictive vector qUAntization. 3. In conventional vector quant1.:tation schemes, the codebooks can be designed using the Linde-Buzo-Gray (LHG) al1,orithm (8) or one of its variations. Even though the LHG algorithm is conceptually and lmplementationally fairly simple, it cannot be used for de~igning optimal codebooks suitable for the SPVQ algorithm. This C1lO be '1een from the fact that the same image vector can be mapped into different code vectors depending on the nature of the adjoining image vectors. As a reqult, it is impossible to obtain a nonoverlapping partition of the training sequence '10 that e'lch subset gets mapped into the ~ame code vector. Since the LHG algorithm re~uireq this type of a partition, it is obvious that the U\G 'llgorithm or any of its variants cannot be used for designing optimal codebooks for the SPVQ method. Even though suboptimal, we have used code books designed using the LBG algorithm in this paper. They are useful mainly for two reasons: a. If the codebook is dense enough, it is possible that the encoJing based on the minimum distortion reconstruction criterion will be different from the minimum distortion encoding of the error sequence and the former encoding will fare much better than the latter approach. Our experience supports this conjecture. h. One big advantage of the LBG :IIKorttho Is its conceptual simplicity. The fact th~t the tralnlnK 'lequcnce can be partitioned Into disjoint ,;ets that map Into I code vector Is v,~ry lI.t~{"t. Thts f:ict enable .. the user to tat l.'r th.~ C,),f,~hD()k to hi,; n,!ed'l. For e~ample, it 1'1 pos'ilhle til cre,lte codebooks with lanter r"pre,;cntat Inn to ed!!,e ph:e lq by ::rerely having a tr,11ninl'( "C'1ue'lce ",tth larger representation of ed~e pixel" {91. The remd lnder of this p,3per is devoted to dlscussln~ so~e experlt:1ental results that demonstr,~te clearly the ability of the SPVQ algorithm to produce high-quality inages at low bit rates. 246 III. EXPERIMENTAL RESULTS The image used for our experiments is termed "woman" and is shown in Fig. 2a. The image consists of 512 x 512 pixels with 8 bits/pixel resolution. The results of encoding the image using the SPVQ algorithm with 32 x 32 sub-blocks, 4 x 4 vectors, 1024 code vectors and 'I = 0.9 is displayed in Fig. 2b. The bit rate for this example (including all the side information) is slightly less than 0.65 bits/pixel. We can see that the visual quality of the reconstructed image is good. Employing a widely-used definition of signal-to-quantization­noise ratio (SQR) [7J given by SQR = mean squared reconstruction error (peak-to-peak value of the image) 2 ' (10) a quantitative measure for the SQR was obtained as 31.3 dB. Here the codebook was obtained using the LBG algorithm with a training sequence consisting of the residuals of seven images other than the "woman" image. IV. SUMMARY AND CONCLUSIONS In this paper we presented a novel approach to image compression using vector quantization of linear (one-step) prediction errors. Results presented in the paper demonstrate the ability of the SPVQ algorithm to produce good quality images at low bit rates. We are at present working on refining our method so as to yield even better results. Some of the areas that are being studied are the design of optimal code books for the SPVQ algorithm, the "constrained" predictor design for more complex structures, improved coding of edge pixels, incorporation of visual models into the data compression algorithm and further Simplifications and improvements in the SPVQ encoder structure. 1. 2. 3. 4. 5. REFERENCES R. L. Baker. Vector quantization of digital images, Ph.D. Thesis, Stanford University, June 198~ --- R. L. Baker and R. M. Gray, "Image compression using non-adaptive spatial vector quantization," Proceedings of the 16th Asilomar Conference on Circuits,--­S" YS'tems, and Computers, 1982. R. L. Baker and R. M. Gray, "Differential vector quantization of achromatic imagery," Proceedings of the International Picture £,oding SympoSTu;;:-pp. 105-106, March i983. R. J. Clarke,Transform Coding of Images, Academic Press, London, 1985. - --- V. Cuperman and A. Gersho, "Vector predictive coding of speech at 16 kbits/s; ~ Transactions on Communications, Vol. cml-33, No.7, pp. 685-696, July 1985. 6. R. M. Gray, "Vector quantization," IEEE ASSP Magazine, Vol. 1, No.2, pp. 4-29, April---- 1984. 7. A. K. Jain, "Image data compression: A review," Proceedings of the IEEE, Vol. 679, No.3, pp. 349-389, March 1981. 8. Y. Linde, A. Buzo, and R. M. Gray, "An algorithm for vector quantizer design," IEEE Transactions on Communications, Vol. COM-28, No. I, pp. 84-95, January 1980. + y,(n ,m) I+q e.'. (n.,< mi)" , I z(n .m) ---'~;)----(- a. ENCODER 9. B. Ramamurthi and A. Gersho, "Image coding using segmented codebooks," Proceedings International Picture Coding Symposium pp. 44-45, March 1983. 10. M. R. Schroeder and B. S. Atal, "Code­excited linear prediction (CELP): High quality speech at very low bit rates," Proceedings ICASSP 1985, pp. 937-940, Tampa, Florida, 11arcli'l985-.-- + + DECODER e(n,m) eq(n,m) '1 I I I y (n.m) q Fig. 1. Inverse x(n.m) >1 Filter + y(n.m) --~~+~----~--; b. ENCODER DECODER z(n,m) z(n.m) Predictive vector quantization algorithm presented a. Traditional predictive quantization. h. in this paper. In actual implementation of the encoder, computed. The inputs to the vector quantizer are y(n,m) the prediction error sequences are not and the code vectors. Fig. 2. a. Original "woman" image. b. Quantized "woman" (0.6484 bits/pixel) 247