# Application of lattice basis reduction algorithms for polynomial approximation with finite wordlength coefficients

Andraž Božiček (2011) Application of lattice basis reduction algorithms for polynomial approximation with finite wordlength coefficients. MSc thesis.

 Preview
PDF
Download (2329Kb)

## Abstract

Lattice basis reduction is a concept for solving diverse problems. Lattice basis reduction algorithms have been successfully used in many areas, including practical applications such as cryptography, GPS navigation and wireless communications. In this master's thesis we have used lattice basis reduction algorithms to solve the problem of designing optimal linear phase finite impulse response filters (FIR filters) with finite wordlength coefficients. The problem was first formulated as the problem of minimax polynomial approximation where the function we try to approximate is represented by the desired frequency response of the filter. The quality of the approximation is measured by minimum of the maximum absolute error or deviation from the desired frequency response. We have used Babai's nearest plane algorithm in the filter design. Babai's algorithm solves the closest vector problem and it uses the basis reduced by the LLL algorithm as an input. Both LLL and Babai's algorithm give result relative to the L2 norm. Optimal filter coefficients are given by the polynomial approximation in L∞ norm, but only algorithms with exponential complexity are available for solving this problem in L∞ norm. We have used algorithms which solve the problem in L2 norm and then added heuristics which improves the results relative to L∞ norm. We also tried a method where the problem was formulated as a system of diophantine equations with lower and upper bound which was then attempted to solve by using the LLL algorithm. Both mentioned methods of filter design were tested on different sets of lowpass and bandstop filters with different frequency-domain specifications. Unfortunately, we didn't get any result from solving the system of diophantine equations. So we have only compared Babai's algorithm and heuristics with two existing design methods which do not use lattice basis reduction algorithms. The first method is simple and is based on rounding infinite precision coefficients obtained by Remez algorithm. This method gives suboptimal results. The second method is based on integer programming, which always gives optimal results but is slow and sometimes the problem is not solved in a reasonable amount of time. Design method with Babai's algorithm and heuristics has been proved in tests to be fast and for almost all sets of filters better than rounding method. In certain cases we have obtained the same results as with integer programming, but unfortunately not always. The set of filters where this method was not proved to be good was the set of filters that have different weight of the error in each band. That is also the major drawback of using Babai's algorithm and heuristics for designing optimal FIR filter with finite wordlength coefficients. In the future the research should be focused on using the lattice basis reduction algorithms with integer programming. The LLL algorithm has been already used to derive a lower bound for approximation error which increases the speed of the branch-and-bound algorithm.

Item Type: Thesis (MSc thesis)
Keywords: lattice basis reduction, FIR filter design, finite wordlength coefficients, LLL algorithm, closest vector problem
Number of Pages: 53
Language of Content: Slovenian
Mentor / Comentors:
Name and SurnameIDFunction
prof. dr. Dušan Kodek236Mentor
Link to COBISS: http://www.cobiss.si/scripts/cobiss?command=search&base=50070&select=(ID=00008629332)
Institution: University of Ljubljana
Department: Faculty of Computer and Information Science
Item ID: 1499
Date Deposited: 13 Sep 2011 14:10
Last Modified: 26 Sep 2011 16:49
URI: http://eprints.fri.uni-lj.si/id/eprint/1499

### Actions (login required)

 View Item