Advanced Textbooks in Control and Signal Processing Series Editors Professor Michael J. Grimble, Professor of Industrial Systems and Director Professor Michael A. Johnson, Professor of Control Systems and Deputy Director Industrial Control Centre, Department of Electronic and Electrical Engineering, University of Strathclyde, Graham Hills Building, 50 George Street, Glasgow G1 1QE, U.K. Other titles published in this series: Genetic Algorithms K.F. Man, K.S. Tang and S. Kwong Neural Networks for Modelling and Control of Dynamic Systems M. Nørgaard, O. Ravn, L.K. Hansen and N.K. Poulsen Modelling and Control of Robot Manipulators (2nd Edition) L. Sciavicco and B. Siciliano Fault Detection and Diagnosis in Industrial Systems L.H. Chiang, E.L. Russell and R.D. Braatz Soft Computing L. Fortuna, G. Rizzotto, M. Lavorgna, G. Nunnari, M.G. Xibilia and R. Caponetto Statistical Signal Processing T. Chonavel Discrete-time Stochastic Processes (2nd Edition) T. Söderström Parallel Computing for Real-time Signal Processing and Control M.O. Tokhi, M.A. Hossain and M.H. Shaheed Multivariable Control Systems P. Albertos and A. Sala Control Systems with Input and Output Constraints A.H. Glattfelder and W. Schaufelberger Analysis and Control of Non-linear Process Systems K. Hangos, J. Bokor and G. Szederkényi Model Predictive Control (2nd Edition) E.F. Camacho and C. Bordons Digital Self-tuning Controllers V. Bobál, J. Böhm, J. Fessl and J. Macháček Control of Robot Manipulators in Joint Space R. Kelly, V.Santibáñez and A. Loría Publication due July 2005 Robust Control Design with MATLAB® D.-W. Gu, P.H. Petkov and M.M. Konstantinov Publication due September 2005 Active Noise and Vibration Control M.O. Tokhi Publication due September 2005 A. Zaknich Solutions Manual for Principles of Adaptive Filters and Self-learning Systems With 6 Figures 123 Anthony Zaknich Ph.D. Centre for Intelligent Information Processing Systems, Department of Electrical and Electronic Engineering, The University of Western Australia, 35 Stirling Highway, Crawley, WA 6007, Australia Apart from any fair dealing for the purposes of research or private study, or criticism or review, as permitted under the Copyright, Designs and Patents Act 1988, this publication may only be reproduced, stored or transmitted, in any form or by any means, with the prior permission in writing of the publishers, or in the case of reprographic reproduction in accordance with the terms of licences issued by the Copyright Licensing Agency. Enquiries concerning reproduction outside those terms should be sent to the publishers. Advanced Textbooks in Control and Signal Processing ISSN 1439-2232 ISBN 1-85233-984-5 Springer Science+Business Media springeronline.com © Springer-Verlag London Limited 2005 The use of registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant laws and regulations and therefore free for general use. The publisher makes no representation, express or implied, with regard to the accuracy of the information contained in this book and cannot accept any legal responsibility or liability for any errors or omissions that may be made. Typesetting: Camera ready by authors Printed in Germany 69/3830-543210 Printed on acid-free paper SPIN 10978566 Preface This Solutions Manual contains the answers along with the problems found at the end of relevant chapters of the book “Adaptive Filters and Self-learning Systems.” It also has a representative course outline, sample assignments and representative past examination papers with solutions that may aid in the design and conduct of such a course. Anthony Zaknich Perth Western Australia January 2004 Contents 2 Linear Systems and Stochastic Processes.......................................................1 2.12 Exercises...................................................................................................1 2.12.1 Problems......................................................................................1 2.12.2 Answers.......................................................................................3 3 Optimisation and Least Square Estimation...................................................9 3.14 Exercises...................................................................................................9 3.14.1 Problems......................................................................................9 3.14.2 Answers.....................................................................................10 4 Parametric Signal and System Modelling....................................................15 4.5 Exercises.................................................................................................15 4.5.1 Problems....................................................................................15 4.5.2 Answers.....................................................................................16 5 Optimum Wiener Filter..................................................................................27 5.4 Exercises.................................................................................................27 5.4.1 Problems....................................................................................27 5.4.2 Answers.....................................................................................28 6 Optimal Kalman Filter...................................................................................33 6.5 Exercises.................................................................................................33 6.5.1 Problems....................................................................................33 6.5.2 Answers.....................................................................................34 7 Power Spectral Density Analysis...................................................................37 7.4 Exercises.................................................................................................37 7.4.1 Problems....................................................................................37 7.4.2 Answers.....................................................................................38 8 Adaptive Finite Impulse Response Filters....................................................43 8.4 Exercises.................................................................................................43 8.4.1 Problems....................................................................................43 8.4.2 Answers.....................................................................................44 viii Contents 9 Frequency Domain Adaptive Filters.............................................................47 9.2 Exercises.................................................................................................47 9.2.1 Problems....................................................................................47 9.2.2 Answers.....................................................................................47 10 Adaptive Volterra Filters...............................................................................51 10.6 Exercises.................................................................................................51 10.6.1 Problems....................................................................................51 10.6.2 Answers.....................................................................................51 12 Introduction to Neural Networks..................................................................53 12.3 Exercises.................................................................................................53 12.3.1 Problems....................................................................................53 12.3.2 Answers.....................................................................................54 Appendix...............................................................................................................57 A.1 Adaptive Systems Course Outline..........................................................57 A.1.1 Lecturer.....................................................................................57 A.1.2 Course Objectives.....................................................................57 A.1.3 Prerequisites..............................................................................58 A.1.4 Assessment................................................................................58 A.1.5 Texts and Recommended Reading List.....................................58 A.1.6 Lectures and Tutorials...............................................................59 A.2 Project Assignments...............................................................................60 A.2.1 Assignment 1.............................................................................60 A.2.1.1 Answer to Assignment 1...........................................61 A.2.2 Assignment 2.............................................................................61 A.2.3 Assignment 3.............................................................................62 A.2.3.1 Answer to Assignment 3...........................................63 A.2.4 Assignment 4.............................................................................64 A.3 Examination Papers................................................................................65 A.3.1 Examination Paper 1.................................................................65 A.3.1.1 Examination Paper 1 Answers..................................69 A.3.2 Examination Paper 2.................................................................78 A.3.2.1 Examination Paper 2 Answers..................................81 2. Linear Systems and Stochastic Processes 2.12 Exercises The following Exercises identify some of the basic ideas presented in this Chapter. 2.12.1 Problems 2.1. Does the equation of a straight line y = α x + β, where α and β are constants, represent a linear system? Show the proof. 2.2. Show how the z-Transform can become the DFT. 2.3. Which of the FIR filters defined by the following impulse responses are linear phase filters? a. h[n] = {0.2, 0.3, 0.3, 0.2} b. h[n] = {0.1, 0.2, 0.2, 0.1, 0.2, 0.2} c. h[n] = {0.2, 0.2, 0.1, 0.1, 0.2, 0.2} d. h[n] = {0.05, 0.15, 0.3, -0.15,-0.15} e. h[n] = {0.05, 0.3, 0.0, -0.3, -0.05} 2.4. Given the following FIR filter impulse responses what are their H(z) and H(z-1)? What are the zeros of the filters? Express the transfer functions in terms of zeros and poles? Prove that these filters have linear phases. a. h[n] = {0.5, 0.5} b. h[n] = {0.5, 0.0, -0.5} c. h[n] = {0.5, 0.0, 0.5} d. h[n] = {0.25, -0.5, 0.25} 2.5. Which of the following vector pairs are orthogonal or orthonormal? a. [1, -3, 5]T and [-1, -2, -1]T 2 Solutions Manual for Adaptive Filters and Self-learning Systems b. [0.6, 0.8]T and [4, -3]T c. [0.8, 0.6]T and [0.6, -0.8]T d. [1, 2, 3]T and [4, 5, 6]T 2.6. Which of the following matrices are Toeplitz? 3 2 1 1 2 3 1 1 1 3 2 1 1 2 3           a. 4 3 2 , b. 1 2 3 , c. 1 1 1 , d. 2 1 2 , e. 2 1 2           5 4 3 1 2 3 1 1 1 1 2 3 3 2 1  1 (1+ j) (1− j)    f. (1− j) 1 (1+ j)   (1+ j) (1− j) 1  What is special about matrices c, d, e, and f? 2.7. Which of the following matrices are orthogonal? Compute the matrix inverses of those that are orthogonal. 0 1 0 0 0 1 1 1 1 1 0 0 1 2 3           a. 0 0 1 , b. 0 1 0 , c. 1 1 1 , d. 0 2 0 , e. 2 1 2           1 0 0 1 0 0 1 1 1 0 0 3 3 2 1 2.8. Why are ergodic processes important? 2.9. Find the eigenvalues of the following 2 x 2 Toeplitz matrix, a b A=  b a Find the eigenvectors for a = 4 and b = 1. 2.10. Compute the rounding quantization error variance for an Analogue to Digital Converter (ADC) with a quantization interval equal to ∆. Assume that the signal distribution is uniform and that the noise is stationary white noise. 2.11. What is the mean and autocorrelation of the random phase sinusoid defined by, x[n]= Asin(nω +φ), given that A and ω are fixed constants 0 0 and φ is a random variable that is uniformly distributed over the interval −π to π. The probability density function for φ is,