NETWORKED MULTISENSOR DECISION AND ESTIMATION FUSION Based on Advanced Mathematical Methods NETWORKED MULTISENSOR DECISION AND ESTIMATION FUSION Based on Advanced Mathematical Methods Yunmin Zhu Jie Zhou Xiaojing Shen Enbin Song Yingting Luo Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Group, an informa business MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accu- racy of the text or exercises in this book. This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software. CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 First issued in hardback 2019 © 2013 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works ISBN-13: 978-1-4398-7452-3 (hbk) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, micro- filming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://www. copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750- 8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identi- fication and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Networked multisensor decision and estimation fusion : based on advanced mathematical methods / Yunmin Zhu … [et al.]. p. cm. Summary: “Multisource information fusion has become a crucial technique in areas such as sensor networks, space technology, air traffic control, military engineering, communications, industrial control, agriculture, and environmental engineering. Exploring recent signficant results, this book presents essential mathematical descriptions and methods for multisensory decision and estimation fusion. It covers general adapted methods and systematic results, includes computer experiments to support the theoretical results, and fixes several popular but incorrect results in the field”-- Provided by publisher. Includes bibliographical references and index. ISBN 978-1-4398-7452-3 (hardback) 1. Sensor networks. 2. Multisensor data fusion--Mathematics. I. Zhu, Yunmin, 1944- TK7872.D48N47 2012 681’.2--dc23 2012015651 Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com Contents Preface ........................................................................ xiii Acknowledgment ............................................................. xix 1 Introduction............................................................... 1 1.1 FundamentalProblems ................................................ 2 1.2 CoreofFundamentalTheoryandGeneralMathematicalIdeas...... 3 1.3 ClassicalStatisticalDecision........................................... 4 1.3.1 BayesDecision ................................................. 5 1.3.2 Neyman–PearsonDecision .................................... 8 1.3.2.1 Neyman–PearsonCriterion .......................... 8 1.3.3 MinimaxDecision ............................................. 10 1.4 LinearEstimationandKalmanFiltering .............................. 11 1.5 BasicsofConvexOptimization........................................ 17 1.5.1 ConvexOptimization.......................................... 17 1.5.1.1 BasicTerminologyofOptimization.................. 17 1.5.2 Duality ......................................................... 22 1.5.3 Relaxation ...................................................... 24 1.5.3.1 S-ProcedureRelaxation............................... 24 1.5.3.2 SDPRelaxation....................................... 26 2 ParallelStatisticalBinaryDecisionFusion .............................. 29 2.1 OptimalSensorRulesforBinaryDecisionGivenFusionRule....... 30 2.1.1 FormulationforBayesBinaryDecision ....................... 30 2.1.2 FormulationofFusionRulesviaPolynomials ofSensorRules................................................. 31 2.1.3 Fixed-PointTypeNecessaryConditionfortheOptimal SensorRules ................................................... 33 2.1.4 FiniteConvergenceoftheDiscretizedAlgorithm............. 37 2.2 UnifiedFusionRule ................................................... 45 2.2.1 ExpressionoftheUnifiedFusionRule........................ 45 2.2.2 NumericalExamples........................................... 48 2.2.2.1 TwoSensors ......................................... 48 v vi ■ Contents 2.2.2.2 ThreeSensors........................................ 50 2.2.2.3 FourSensors ......................................... 52 2.3 ExtensiontoNeyman–PearsonDecision.............................. 53 2.3.1 AlgorithmSearchingforOptimalSensorRules............... 56 2.3.2 NumericalExamples........................................... 57 3 GeneralNetworkStatisticalDecisionFusion............................ 59 3.1 ElementaryNetworkStructures....................................... 60 3.1.1 ParallelNetwork............................................... 60 3.1.2 TandemNetwork.............................................. 62 3.1.3 Hybrid(Tree)Network........................................ 64 3.2 FormulationofFusionRuleviaPolynomialsofSensorRules ........ 64 3.3 Fixed-PointTypeNecessaryConditionforOptimal SensorRules ........................................................... 69 3.4 IterativeAlgorithmandConvergence ................................. 71 3.5 UnifiedFusionRule ................................................... 74 3.5.1 UnifiedFusionRuleforParallelNetworks.................... 75 3.5.2 UnifiedFusionRuleforTandemand HybridNetworks.............................................. 78 3.5.3 NumericalExamples........................................... 79 3.5.3.1 Three-SensorSystem................................. 80 3.5.3.2 Four-SensorSystem.................................. 82 3.6 OptimalDecisionFusionwithGivenSensorRules................... 84 3.6.1 ProblemFormulation.......................................... 85 3.6.2 ComputationofLikelihoodRatios............................ 87 3.6.3 LocallyOptimalSensorDecisionRules withCommunicationsamongSensors ........................ 88 3.6.4 NumericalExamples........................................... 90 3.6.4.1 Two-SensorNeyman–Pearson DecisionSystem ..................................... 91 3.6.4.2 Three-SensorBayesianDecisionSystem ............ 91 3.7 SimultaneousSearchforOptimalSensorRulesand FusionRule............................................................ 96 3.7.1 ProblemFormulation.......................................... 96 3.7.2 NecessaryConditionsforOptimalSensorRules andanOptimalFusionRule .................................. 99 3.7.3 IterativeAlgorithmandItsConvergence...................... 103 3.7.4 ExtensionstoMultiple-BitCompressionandNetwork DecisionSystems .............................................. 110 3.7.4.1 ExtensionstotheMultiple-BitCompression........ 110 3.7.4.2 ExtensionstoHybridParallelDecisionSystem andTreeNetworkDecisionSystem................. 112 3.7.5 NumericalExamples........................................... 116 Contents ■ vii 3.7.5.1 TwoExamplesforAlgorithm3.2.................... 116 3.7.5.2 AnExampleforAlgorithm3.3 ...................... 119 3.8 PerformanceAnalysisofCommunicationDirection forTwo-SensorTandemBinaryDecisionSystem .................... 120 3.8.1 ProblemFormulation.......................................... 122 3.8.1.1 SystemModel........................................ 122 3.8.1.2 BayesDecisionRegionofSensor2.................. 122 3.8.1.3 BayesDecisionRegionofSensor1 (FusionCenter)...................................... 127 3.8.2 BayesCostFunction........................................... 128 3.8.3 Results ......................................................... 129 3.8.4 NumericalExamples........................................... 140 3.9 NetworkDecisionSystemswithChannelErrors ..................... 143 3.9.1 SomeFormulationsaboutChannelError..................... 144 3.9.2 NecessaryConditionforOptimalSensorRulesGiven aFusionRule .................................................. 145 3.9.3 SpecialCase:MutuallyIndependentSensor Observations................................................... 149 3.9.4 UnifiedFusionRulesforNetworkDecisionSystems......... 151 3.9.4.1 NetworkDecisionStructureswith ChannelErrors....................................... 151 3.9.4.2 UnifiedFusionRuleinParallelBayesianBinary DecisionSystem ..................................... 154 3.9.4.3 UnifiedFusionrulesforGeneralNetwork DecisionSystemswithChannelErrors.............. 155 3.9.5 NumericalExamples........................................... 157 3.9.5.1 ParallelBayesianBinaryDecisionSystem........... 157 3.9.5.2 Three-SensorDecisionSystem ...................... 159 4 SomeUncertainDecisionCombinations ................................ 163 4.1 RepresentationofUncertainties....................................... 164 4.2 DempsterCombinationRuleBasedonRandom SetFormulation ....................................................... 165 4.2.1 Dempster’sCombinationRule................................ 167 4.2.2 MutualConversionoftheBasicProbabilityAssignment andtheRandomSet........................................... 167 4.2.3 CombinationRulesoftheDempster–ShaferEvidencesvia RandomSetFormulation...................................... 168 4.2.4 AllPossibleRandomSetCombinationRules................. 169 4.2.5 CorrelatedSensorBasicProbabilisticAssignments............ 171 4.2.6 OptimalBayesianCombinationRule......................... 172 4.2.7 ExamplesofOptimalCombinationRule ..................... 174 viii ■ Contents 4.3 FuzzySetCombinationRuleBasedonRandomSet Formulation ........................................................... 177 4.3.1 MutualConversionoftheFuzzySetandthe RandomSet.................................................... 178 4.3.2 SomePopularCombinationRulesofFuzzySets.............. 179 4.3.3 GeneralCombinationRules................................... 181 4.3.3.1 UsingtheOperationsofSetsOnly.................. 182 4.3.3.2 UsingtheMoreGeneralCorrelationofthe RandomVariables ................................... 183 4.3.4 Relationshipbetweenthet-NormandTwo-Dimensional DistributionFunction ......................................... 184 4.3.5 Examples....................................................... 186 4.4 HybridCombinationRuleBasedonRandomSet Formulation ........................................................... 188 5 ConvexLinearEstimationFusion........................................ 191 5.1 LMSEEstimationFusion.............................................. 192 5.1.1 FormulationofLMSEFusion................................. 192 5.1.2 OptimalFusionWeights ...................................... 195 5.2 EfficientIterativeAlgorithmforOptimalFusion..................... 200 5.2.1 AppropriateWeightingMatrix................................ 201 5.2.2 IterativeFormulaofOptimalWeightingMatrix.............. 204 5.2.3 IterativeAlgorithmforOptimalEstimationFusion........... 205 5.2.4 Examples....................................................... 210 5.3 RecursionofEstimationErrorCovarianceinDynamicSystems ..... 212 5.4 OptimalDimensionalityCompressionforSensorDatain EstimationFusion ..................................................... 214 5.4.1 ProblemFormulation.......................................... 215 5.4.2 Preliminary..................................................... 216 5.4.3 AnalyticSolutionforSingle-SensorCase...................... 218 5.4.4 SearchforOptimalSolutionintheMultisensorCase ........ 220 5.4.4.1 ExistenceoftheOptimalSolution................... 220 5.4.4.2 OptimalSolutionataSensorWhileOtherSensor CompressionMatricesAreGiven ................... 221 5.4.5 NumericalExample............................................ 223 5.5 QuantizationofSensorData .......................................... 224 5.5.1 ProblemFormulation.......................................... 227 5.5.2 NecessaryConditionsforOptimalSensorQuantization RulesandOptimalLinearEstimationFusion................. 229 5.5.3 Gauss–SeidelIterativeAlgorithmforOptimalSensor QuantizationRulesandLinearEstimationFusion............ 235 5.5.4 NumericalExamples........................................... 237 Contents ■ ix 6 KalmanFilteringFusion.................................................. 241 6.1 DistributedKalmanFilteringFusionwithCross-Correlated SensorNoises .......................................................... 243 6.1.1 ProblemFormulation.......................................... 244 6.1.2 DistributedKalmanFilteringFusionwithoutFeedback...... 246 6.1.3 OptimalityofKalmanFilteringFusionwithFeedback....... 249 6.1.3.1 GlobalOptimalityoftheFeedback FilteringFusion...................................... 250 6.1.3.2 LocalEstimateErrors................................ 251 6.1.3.3 TheAdvantagesoftheFeedback .................... 252 6.2 DistributedKalmanFilteringFusionwithSingularCovariances ofFilteringErrorandMeasurementNoises........................... 254 6.2.1 EquivalenceFusionAlgorithm ................................ 255 6.2.2 LMSEFusionAlgorithm ...................................... 255 6.2.3 NumericalExamples........................................... 257 6.3 OptimalKalmanFilteringTrajectoryUpdatewithUnideal SensorMessages ....................................................... 261 6.3.1 OptimalLocal-ProcessorTrajectoryUpdatewithUnideal Measurements.................................................. 262 6.3.1.1 OptimalLocal-ProcessorTrajectoryUpdatewith AdditionofOOSMs................................. 263 6.3.1.2 OptimalLocal-ProcessorTrajectoryUpdatewith RemovalofEarlierMeasurement.................... 267 6.3.1.3 OptimalLocal-ProcessorTrajectoryUpdatewith SequentiallyProcessingUnidealMeasurements..... 268 6.3.1.4 NumericalExamples................................. 269 6.3.2 OptimalDistributedFusionTrajectoryUpdate withLocal-ProcessorUnidealUpdates........................ 271 6.3.2.1 OptimalDistributedFusionTrajectoryUpdate withAdditionofLocalOOSMUpdate............. 272 6.3.2.2 OptimalDistributedStateTrajectoryUpdate withRemovalofEarlierLocalEstimate............. 274 6.3.2.3 OptimalDistributedFusionTrajectoryUpdate withSequentialProcessingofLocal UnidealUpdates..................................... 275 6.4 RandomParameterMatricesKalmanFilteringFusion ............... 276 6.4.1 RandomParameterMatricesKalmanFiltering ............... 276 6.4.2 RandomParameterMatricesKalmanFiltering withMultisensorFusion....................................... 278 6.4.3 SomeApplications............................................. 281 6.4.3.1 ApplicationtoDynamicProcesswith FalseAlarm........................................... 281