University of Southern California Dissertations and Theses

Iterative data detection: complexity reduction and applications

(USC Thesis Other)

Iterative data detection: complexity reduction and applications

PDF

Download a page range

Download transcript

Copy asset link

Request this asset

Transcript (if available)

Content ITERATIVE DATA DETECTION: COMPLEXITY REDUCTION AND APPLICATIONS by Xiaopeng Chen A D issertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In P artial Fulfillment of the Requirements for the Degree D O CTOR OF PHILOSOPHY (Electrical Engineering) December 1999 (c) Copyright by Xiaopeng Chen, 1999 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UNIVERSITY OF SOUTHERN CALIFORNIA THE GRADUATE SCHOOL UNIVERSITY PARK LOS ANGELES. CALIFORNIA 90 00 7 This dissertation, written by ' X ia o jje n g C h e n ............................................................ under the direction of h..x& .................. Dissertation Committee, and approved by all its members, has been presented to and accepted by The Graduate School, in partial fulfillment of re quirements for the degree of DOCTOR OF PHILOSOPHY Dean o f Graduate Studies Chairperson Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. This dissertation is dedicated to m y parents, Chen Yu Su and Zou X iu Lin, fo r their endless love and continuing encouragem ent . . . ii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. A cknow ledgem ents I would like to express my deepest appreciation to my PhD thesis advisor Prof. Keith M. Chugg for his inspiration, encouragement and friendship provided throughout this research. I am grateful to him to provide me the freedom to explore new areas of interest and the opportunity to explore my research potential. The two and half years staying with him have changed me and my career forever. My deepest appreciation is extended to Prof. Alan E. VVillner for the joyful cooperation and his kind understanding. I would like to thank Prof. Antonio Orgeta for the joyful cooperation and for serving on my dissertation committee. I also would like to thank Prof. Mark A. Xeifeld from University of Arizona for the fruitful joint research on the POM systems. Sincere thanks are extended to Prof. Kenneth Alexander for his lecturing and for serving on my dissertation committee. Thanks also go to Prof. Lloyd R. Welch and Prof. Zhen Zhang for their lecturing and serving on my qualifying exam committee. I wish to thank all of my friends in the Communication Sciences Institute (CSI) for their friendship, and for fruitful technical and casual discussions. In particular. I would like to thank Dr. Gent Paparisto for a chat on TRAM B which changed my life, and to thank my officemates, Dr. Achilleas Anastasopoulos and Carlos J. Corrada. for the one year happy time together. I also would like to thank the cur rent and former members of OcLab: Dr. Kai-Ming Feng. Dr. Jin-Xing Cai. Dr. Xin Jiang. Dr. Yong Xie. Dr. Bogdan Honaca. Dr. Imran Hayee for their help and friend ship. Thanks also go to Dr. W an-Jiun Liao and Ruhua He for the happy chats in EEB507. Thanks to Phunsak Thiennviboon. Jackson Cilia. Lei Zhuge. Guangcai Zhou. Dr. Jeng-Hong Chen and Yuankai Wang for their help and warm friendship. The acknowledgement would not be complete without mentioning the staff of CSI. especially Ms. Milly Montenegro and Ms. Mayumi Thrasher, for their support and adm inistrative assistance. iii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. I am also grateful to my Chinese friends for their precious friendship. In particu lar. I would like to thank Yongliang Han, Cao Guo, Yue Chen, Miao Fu, Feng Zeng, Wei Liang. Rongrong Ren and Xiaohui Lii for those inspiring pseudo-scientific chats around the dinning table. Special thanks to Xiao Zhang for her support during the writing of this dissertation. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. C ontents A cknow ledgem ents iii List O f Tables viii List Of Figures ix List of A bbreviations xi A bstract xii 1 Introduction 1 1.1 Data Detection P ro b lem s................................................................................... 1 1.2 Iterative D ata D e te c tio n ................................................................................... 3 1.3 Research F o c u s .................................................................................................... 5 1.4 Organization ....................................................................................................... 6 2 Soft-In p u t/S oft-O u tp u t A lgorithm s and Iterative D etection 8 2.1 Soft-Input/Soft-O utput A lg o rith m s............................................................. 8 2.1.1 Hard Decision and Soft In fo rm atio n ........................................ 8 2.1.2 The A PP and MSM SISO Algorithm s for Simple FSMs . . . . 10 2.1.2.1 The Simple Finite S tate M a c h in e .................................... 10 2.1.2.2 The APP-SISO Algorithm for a Simple FSM . . . . 13 2.1.2.3 The MSM SISO algorithm for a simple F S M ............... 15 2.1.2.4 APP/M SM -SISOs and a Translation R u l e ................... 17 2.1.3 Categorization and History of SISO A lg o rith m s ................. 18 2.1.4 Code Networks and SISO A lg o rith m s..................................... 21 2.1.5 Bayesian Networks. Extrinsic Information and SISO Algorithm s 23 2.2 Iterative D e te c tio n ............................................................................................ 26 2.2.1 P rin c ip le............................................................................................ 26 2.2.2 Iterative Detection for M ultiple Dimensional S y s te m s ....................27 2.2.3 Iterative Detection for Decomposed S y s te m s ................................ 28 v Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3 Iterative Two D im ensional D ata D etection 29 3.1 Basic Assumptions and Signal M o d e l........................................................... 29 3.2 An Iterative Concatenated D etecto r.............................................................. 31 3.2.1 A Concatenated Channel M o d e l......................................................... 31 3.2.2 2D SISO Algorithms and Iterative D e te c to rs ................................ 32 3.2.3 A pplications.............................................................................................. 36 3.2.3.1 The Minimum Distances for 2D ISI/AW GX Channels 36 3.2.3.2 D ata Detection for Page-Oriented Optical Memory . 39 3.2.3.3 Image H alfto n in g .............................................................. 45 3.3 An Iterative D istributed D e te c to r.................................................................. 50 3.3.1 A Bayesian Network Channel Model .............................................. 50 3.3.2 An Iterative D istributed D e te c to r.................................................... 52 3.3.3 D ata Detection for the 2D ISI/AWGX C h a n n e l ......................... 58 3.3.3.1 Convergence P r o p e r ty .................................................... 58 3.3.3.2 Im pact of Reduced Connection Complexity ................ 59 3.3.3.3 2D ISI M itigation.............................................................. 59 4 Iterative D etection for C om plexity R eduction 61 4.1 Complexity P r o b le m ......................................................................................... 61 4.2 Reduced-State SISO Algorithm ( b r i e f ) ....................................................... 62 4.3 Iterative Detection for Sparse ISI C h an n e ls................................................ 63 4.3.1 Sparse ISI Channel and Existing A pproaches....................................63 4.3.2 The Sparse SISO A lg o rith m .............................................................. 67 4.3.2.1 A Distributed SISO Associated w ith a Single Entry . 67 4.3.2.2 SISO for a Grouped Entry S e t....................................... 68 4.3.2.3 Decision Feedback SISO and Multi-SISO Algorithms 70 4.3.2.4 Comparison of Features and C o m p le x ity .................. 72 4.3.3 Design Rules of the Sparse SISO A lg o rith m s............................... 73 4.3.3.1 The Pivot Entry and the Type of S IS O ..................... 73 4.3.3.2 Convergence Rate of S e lf-Ite ra tio n ............................ 74 4.3.3.3 Im pact of Decision Feedback ........................................... 76 4.3.3.4 Performance Improvement by M u lti-S IS O ............... 78 4.3.3.5 Sum m ary of Design R u le s .............................................. 79 4.3.4 BER Performance of the Sparse S I S O s ......................................... 79 5 Toy Problem s and O pen Problem s 84 5.1 The Intractability of Finding the M L P E ................................................... 84 5.1.1 A Brief Introduction to the Complexity T h e o r y ............................ 84 5.1.2 Decision Problem Related to 2D DLM P r o b le m ............................ 85 5.1.3 The XP-Completeness of 2 D T D ....................................................... 86 5.1.4 On the 2D i n d e x .................................................................................. 86 5.2 Symbol versus Sequence D etection................................................................. 88 5.2.1 High SXR C a s e ..................................................................................... 88 vi Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5.2.2 A/-PSK Signal Case ............................................................................. 91 5.3 On MSM SISO A lg o rith m s.............................................................................. 94 5.3.1 Normalization in MSM SISO A lg o rith m s....................................... 94 5.3.2 Factor Ar 0 in MSM SISO A lg o r ith m s .............................................. 95 5.4 On Distance Spectrum of ISI C h a n n e ls ....................................................... 96 5.4.1 Two Q uadratic Forms for the D is ta n c e .......................................... 96 5.4.2 The "W orst” ISI Channel .................................................................. 99 5.4.3 On the Minimum Error S e q u e n c e ........................................................100 5.4.4 About 2D ISI C hannels............................................................................ 102 5.5 On Regular Sparse ISI C h an n e ls........................................................................102 5.5.1 The Simple Sparse ISI C h an n e ls........................................................... 102 5.5.2 The Regular Sparse ISI C h a n n e ls ........................................................104 5.6 Likelihood Combining ........................................................................................106 5.6.1 A Toy Problem of Joint D etectio n ........................................................106 5.6.2 Likelihood Combining Toy P ro b le m ....................................................109 Appendix A Matrix-based Com putation of the FL-APP-SISO A lg o r ith m ...........................I l l Appendix B Complexity Reduction Techniques for SISO A lg o rith m s..................................... 112 A ppendix C The Equivalence of the FL-A PP and L2V S -A P P ................................................... 113 vii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. List O f Tables 2.1 Translation rule of the APP and MSM versions of a SISO algorithm . 17 2.2 Complexity comparison of F I/F L /F W -S IS O algorithm s.......................... 19 3.1 Param eters for the 2D SISO............................................................................... 34 3.2 Param eters of 2D ISI channels used in sim ulations.........................................40 4.1 Comparison of algorithms for S-ISI channels................................................ 72 4.2 S-ISI channels used in numerical ex p erim en ts............................................ 73 viii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. List O f F igures 1.1 The generic com m unication/storage system m odel..................................... 2 1.2 The readout of a single "1" in the POM with or w ithout memory. . . 2 1.3 Two applications of the iterative detection technique................................ 5 1.4 The organization of this dissertation............................................................... 7 2.1 Com parison of a hard decision and a soft inform ation.............................. 9 2.2 M arginalization and combining operations on the soft information. . . 10 2.3 Param eters defined for a generic finite state m achine................................ 11 2.4 A simple finite state m achine............................................................................ 11 2.5 C om putational architecture of the APP-SISO for a simple FSM. . . . 14 2.6 Inform ation collection ranges of FI/FL /FW -SISO algorithm s.....................19 2.7 Building blocks for the code network and the iterative decoder............. 22 2.8 A cycle-free and non-cycle-free Bayesian networks...................................... 23 2.9 Inform ation flow paths in the BPA.................................................................. 24 2.10 An inform ation theory' view of the "extrinsic” inform ation..................... 25 2.11 A subsystem network and the corresponding iterative detection scheme. 26 3.1 An im plem entation of the MLPE detector using the VA.......................... 30 3.2 A serially concatenated channel m o d e l ....................................................... 32 3.3 A simple 2D SISO ............................................................................................. 33 3.4 A com posite 2D SISO algorithm ...................................................................... 35 3.5 Normalized d !m in and corresponding error patterns for the 3 x 3 2D ISI. 38 3.6 Decision feedback VA for the 2D ISI channel................................................ 42 3.7 BER perform ance of various detectors for Chan-A .......................................... 43 3.8 BER perform ance of various detectors for Chan-B...................................... 44 3.9 BER performance of various detectors for Chan-C...................................... 45 3.10 P rinter halftoned Lena image with 256 gray-levels..................................... 47 3.11 Threshold Halftoned Lena image...................................................................... 48 3.12 Halftoned Lena image using the DFYA.......................................................... 4S 3.13 Halftoned Lena image using the 2D SISO with the uniform initialization. 49 3.14 Halftoned Lena image using the 2D SISO with a random initialization. 49 3.15 Halftoned Lena image using error diffusion................................................... 50 3.16 Halftoned Lena image using a sophisticated iterative scheme................. 51 3.17 A Bayesian network model for the 2D ISI/AWGX channel..................... 52 ix Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3.18 An iterative distributed detector for 2D ISI/AW GX channels....................53 3.19 Block diagram of the 2D4 algorithm .............................................................. 54 3.20 Several pixel sets defined for the 2D4 algorithm ........................................ 55 3.21 Several connection structures defined for the 2D4 algorithm ...................... 57 3.22 Convergence property of the 2D4 algorithm ................................................ 58 3.23 Im pact of the reduced connection complexity............................................. 60 4.1 Hardness of problems.......................................................................................... 62 4.2 Two examples of the sparse ISI channel....................................................... 64 4.3 Neighborhood of a^ for a sparse ISI channel............................................... 66 4.4 A Bayesian network model for a sparse ISI channel................................. 68 4.5 Bayesian network models for an ISI channel............................................... 70 4.6 Performance of an S-SISO with different pivot entries............................. 75 4.7 Convergence property of the S-SISO.............................................................. 76 4.8 Im pact of hard decision feedback on S-SISO............................................... 77 4.9 Performance improvement by using the m ulti-DSISO.............................. 78 4.10 Simulation results for simple S-ISI Channel D ........................................... 80 4.11 Simulation results for S-ISI Channel E (8-PSK )........................................ 81 4.12 Simulation results for S-ISI Channel F ......................................................... 82 4.13 Simulation results for S-ISI Channel G ......................................................... 83 5.1 Bayesian networks embedded in the 2D M LPE problem........................ 87 5.2 A non-equally spaced QPSK constellation................................................... 91 5.3 Bit-by-bit decision regions in various PSK constellations....................... 93 5.4 An equivalent model for a simple S-ISI channel............................................104 5.5 A two dimensional ISI model for a regular S-ISI channel.......................... 105 5.6 Two independent data sequences with one common symbol.................... 106 5.7 D ata on a cross........................................................................................................108 x Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. List of A bbreviations 1(2)D .................................... One(Two) Dimensional 2D4 ........................................ Two Dimensional D istributed D ata Detection APP ...................................... A Posteriori Probability AW GX .................................Additive White Gaussian Xoise B E R .......................................Bit Error Rate BPA ...................................... Belief Propagation Algorithm DF ........................................ Decision Feedback DFE ...................................... Decision Feedback Equalization DFYA ...................................Decision Feedback Viterbi Algorithm DLM .....................................Digital Least Metric FI .......................................... Fixed Interval FL ..........................................Fixed Lag FSM ...................................... Finite State Machine FW ........................................ Fixed Window IDD ...................................... Iterative Data Detection IID ........................................ Independently. Identically D istributed ISI .......................................... Inter-Symbol Interference LUT ...................................... Look-Up Table MLBE ...................................Maximum Likelihood Bit(Symbol) Estim ate MLPE ...................................Maximum Likelihood Page Estim ate MLSE ...................................Maximum Likelihood Sequence Estim ate M M S E .................................. Minimum Mean Square Error MSM .................................... Minimum Sequence Metric XP ........................................ Xondeterministic Polynomial PCCC ...................................Parallel Concatenated Convolutional Code POM .................................... Page-oriented Optimal M emory PSK ...................................... Phase Shift Keyed SCCC ...................................Serially Concatenated Convolutional Code S -IS I...................................... Sparse Inter-Symbol Interference SISO .................................... Soft-Input/Soft-O utput Algorithm SXR ...................................... Signal-to-Xoise Ratio TCM .................................... Trellis Coded M odulation YA ........................................ Viterbi Algorithm Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. A b stract Iterative detection techniques, originally introduced for the decoding of Turbo codes, have become a viable and approxim ately optim al alternative for solving various complicated d ata detection problems. The effectiveness of iterative d ata detection is attributed to the sharing of soft inform ation among the soft-input/soft-output (SISO) processors. Each single SISO processor solves a subproblem of the original problem approxim ately by neglecting dependence on the rest of the system. How to decompose the original problem into local problems directly affects the performance of the corresponding iterative detection. As a specific instance of multiple dimensional systems, the two-dimensional (2D) data detection problem arises from the need for d ata retrieval in page-oriented stor age systems. The existence of a 2D inter-symbol interference (ISI) makes this prob lem extremely difficult. To apply the iterative detection technique, two different de composition schemes are developed for the underlying 2D system. By a colum n/row wise processing, the resulting serially concatenated channel model consists of two one-dimensional (ID ) finite state machines connected by a block interleaver. Cor respondingly. 2D SISO processors are able to retrieve the d ata near-optimally. The same approach is applied to several other 2D applications. On the other hand, by a pixel-wise processing, the resulting lattice channel model is made of an array of ISI processors. The corresponding iterative detector has a fully-distributed compu tational architecture which is well suited for the hardware im plem entation. Although the optim al sequence (ID ) d ata detection can be effectively realized, it can become infeasible in many real-world applications. Iterative detection technique can be used to reduce its complexity significantly. A generic reduced-state SISO algorithm developed by applying decision feedback is introduced briefly. It has the same in p u t/o u tp u t interface as the standard SISO algorithm and therefore is widely applicable. A specific sparse SISO algorithm (S-SISO) is developed for arbitrary xii Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. sparse ISI channels. Its complexity is only determined by the num ber of non-zero taps in the sparse ISI channel. A list of design rules for the S-SISO is provided. Various toy and open problems are also studied to provide the insight into other more com plicated problems. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. C hapter 1 Introduction 1.1 D ata D etection Problem s In many digital communication or storage systems [Ko71], the receiver has access only to a version of the d ata which have been distorted, either intentionally or unin tentionally. by memory in the system. This system memory can come from various sources individually, or simultaneously. For example, in a system in which the mes sage is protected by a convolutional code, the memory is intentionally introduced by the encoder at the transm itter to achieve a better system performance. In a TCM /interleaver/ISI/A W GN (additive white Gaussian noise) channel, the memory comes from both the trellis coded modulation (TCM) encoder used for the perfor mance enhancement and the inter-symbol interference (ISI) caused by the non-ideal channel characteristics. The possible sources of memory in a generic communica tion/storage system are illustrated in Fig. 1.1. Since memory commonly exists in transm ission/storage systems, how to mitigate (exploit) its effect and retrieve the transm itted (stored) data from the observations is a challenging task in the system design. It is interesting to note that some problems, e.g.. certain image process ing problems [ChChOrCh98] which are not originated from communication/storage applications, can be fit into this framework and solved using the same techniques. The system performance may be greatly improved when the receiver uses the memory characteristics of the system constructively. W hen the memory structure of the system is simple, e.g., a convolutional coded system or a Q PSK /ISI channel, one can build a detector that exploits the memory structure effectively. It is well known that the Viterbi algorithm (VA) [Vi67, Fo73] yields the maximum likelihood sequence 1 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. inform ation source source ^ encoder channel d ig ita l ^ e n co d er m o d u lato r memory i research focus: | 1 ( 1 ) channel m odeling 1 I (2) detector design | source channel d ig ita l d e co d er^ ~ decoder dem od tran sm issio n / storage channel inform ation sink Figure 1.1: The generic com m unication/storage system model. estimation (MLSE) for the system in which the em bedded memory- can be modeled as a sequential finite state machine (FSM) [Ra89]. However, when the memory structure becomes more complicated or the size of the memory becomes larger, the complexity of the corresponding data detection problem will rapidly become prohibitive. For example, in a page-oriented optical memory- (POM) system, many sources of memory- and noise arise from the optical system itself. One of the major defects can be modeled as a tyvo dimensional (2D) memory- caused by the physical blurring effect [ChChXe99]. Illustrated in Fig.l.2(left), when the readout system Figure 1.2: The readout of a single "1" in the POM yvith or without memory. does not suffer the blurring defect, a single bit "1" will be observed only at its own location and the optim al estim ate is achievable based on this observation alone. However, in most POM systems, the readout system will pick up (inter-symbol) 2 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. interference from this bit at its surrounding locations because of the blurring effect. A 3 x 3 2D ISI model for a realistic POM system is shown in Fig.l.2(right). Although only a single bit "1" is stored at the center pixel of the page, the receiver observes a group of non-zero outputs. Consequently, the recording/retrieval process in such a POM system can be modeled as a 2D-ISI/AW GX channel. Even when the size of this 2D ISI is small, the lack of natural order of the 2D index makes it hardly solvable by any technique strictly relying on an ordered index system, e.g.. Dijkstra's algorithm [CoLeRi90] works only on a directed tree. Although some traditional approaches. e.g.. threshold detection and minimum means square error equalization [ChChXe97|. can be readily adapted to this 2D system, they usually perform much worse than the best achievable performance. Attem pts have been made to adapt some sequential algorithms to solve the 2D problem. In [HeGuHe96], the VA is run for each row with the help of hard decisions fed back from the previous rows. Unlike the MLSE which is effectively realized by the VA for the sequence detection, the corresponding maximum likelihood page estim ation (MLPE) associated with the 2D ISI/AW GX channel has not been able to be accomplished by any efficient way. On the other hand, the memory in some systems may have a simple structure but a huge size. For example, in high-data-rate digital communication systems the memory of an ISI can be very long. It is well known that the VA has a complexity exponential to the length of the ISI. Once again, the traditional detection tech niques become prohibitively complex. Many complexity reduction techniques, e.g.. decision feedback [DuHe89] and state reduction [EyQu88], have been widely applied to simplify the corresponding detectors. However, their applicability is constrained to certain channel characteristics. For example, when the underlying channel is a non-minimum-phase ISI channel, the system performance may degrade significantly by applying the delayed decision feedback sequence detector in [DuHe89] or the reduced-state sequence detector in [EyQu88]. 1.2 Iterative D ata D etectio n The invention of the Turbo coding in 1993 [BeGlTh93] is a m ajor breakthrough in the field of inform ation and communication theory. The success of Turbo codes mainly relies on the iterative decoding (also called Turbo decoding) technique. When 3 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. this scheme is applied to d ata detection problems, we call it iterative data detec tion (IDD). Generally speaking, the IDD technique consists of two components: the soft-input/soft-output (SISO) algorithm and the iterative algorithm . The SISO al gorithm acts as a soft information processor which can refine the belief (soft output) based on the belief (soft input) fed from other processors and the local evidence (i.e.. the structure of the underlying local FSM and/or observations). The iterative algorithm provides a way to exchange and share the soft inform ation among pro cessors in an effective manner. The resulting system performance usually is much better than traditional approaches. At the same time, the complexity of the IDD technique is still acceptable for hardware/software im plem entations. Another m ajor advantage of the IDD technique is th at it is so flexible th a t it is applicable to vari ous problems in com m unication/storage systems, e.g., decoding of (both serial and parallel) Turbo codes [BeGlTh93, BeMoDiPo96] and other error-correction codes [HaOfPa96. LuBoBr98]. near-capacity multi-user detection [Mo98], the detection of TCM signals in fading channels [AnCh98]. For a b etter understanding of the IDD technique, two examples are illustrated in Fig.1.3. The first one is the original Turbo code presented in [BeGlTh93]. This Turbo code consists of two recursive system atic convolutional sub-codes (RCC). Each of them has a small memory. They are separated by a pseudo-random interleaver. In the receiver, the Turbo decoder runs two SISO algorithms, each of which exploits the memory structure of one subcode based on proper observations. By iteration, these two SISO modules exchange their outputs for several times. The system performance has been shown to be only ldB from the theoretical limit predicted by Shannon. In the second example (Fig. 1.3(b)), rc users are simultaneously sending their own messages, which are protected by a interleaver/feedforward-error-correction code (FEC), through a single channel. This is a multi-user communication system. The receiver uses n SISO modules, one for each user, to decode messages. Then the soft inform ation from a specific SISO module is iterative!}' shared by other SISO modules. Consequently, the single-user performance can be achieved asymptotically in a highly correlated multiuser systems [Mo98]. From these two examples, we see that the underlying systems have been decomposed in some way (according to the memory structure) into a set of dependent subsystems. The IDD technique solves 4 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. in p u t RCC 1 i:S IS O l< v a w g n , k o u tp u t > Interleaver ^ R C C 2 ^ S I S 0 2 Turbo encoder T urbo decoder (a) a Turbo code ^S IS O 1 > d e sr s°o d / ~ SIS° 2 ^S IS O n user 1 »~FEC user 2 ^ F E C user n >~FEC (b) iterative m ultiuser detection Figure 1.3: Two applications of the iterative detection technique. the local problem in each subsystem by applying a proper SISO algorithm, and exploits the dependence among the subsystems by using the iterative algorithm. 1.3 R esearch Focus In many applications to which the IDD technique is applicable, the underlying systems are self-organized into simple subsystems. For example, in Turbo codes (Fig. 1.3(a)) or more general code networks [BeMoDiPo98], each constituent code is a subsystem with simple memory structures. In the m ulti-user detection applica tion (Fig.1.3(b)). each user is naturally treated as a subsystem . In many real-world applications, however, the underlying systems {e.g., 2D ISI/AW GX channel) may not naturally consist of smaller systems. In order to apply the IDD technique, we must first decompose these systems. Successful application of the IDD technique in these applications requires finding a proper decomposition scheme. As illustrated in Fig. 1.1. the research focus of this dissertation is to decompose the given channels o Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. properly and develop corresponding SISO algorithm s for the IDD. In this disserta tion. the IDD technique will be used to solve some difficult d a ta detection problems which arise from the real-world com m unication/storage systems. 1.4 O rganization In C hapter 2. two corner stones of the IDD technique - SISO algorithm and iterative detection - are discussed. First, the APP-SISO and MSM-SISO algorithms are developed for a simple FSM without fixing the boundary of observations employed, based on which the fixed interval (FI), fixed lag (FL) [ChCh98b] and fixed window (F\V) [ChCh99] versions of the APP/M SM -SISO algorithm are defined. A brief comparison of the APP-SISO and the MSM-SISO is given and a generic translation rule between the probability-based and metric-based SISO algorithm s is established. The rich history of SISO algorithm is also briefly discussed. Then the SISO algorithm is discussed in more general underlying systems, i.e., code networks [Be\IoDiPo98] and Bayesian networks [Pe86]. The concept of iterative detection is discussed in the domain of multiple-dimensional systems and decomposed systems. In C hapter 3. the IDD technique is applied to solve the d ata detection problem of which the underlying system is 2D [ChChXe99]. First, a generic serially con catenated channel model is developed for the 2D system. Then a simple 2D SISO processor and a composite 2D SISO processor [ChCh98] are developed. These it erative d ata detectors work close to the theoretical perform ance bounds in the 2D ISI/AWGX system . They are also used to find the constrained minimum distance for the 2D ISI and to halftone gray-level images in the sense of a least metric cost [ChChOrCh98. ThChChOr99j. Another channel model consists of an array of identi cal ISI processors for each of which a SISO processor can be built [ChChXe98]. This iterative detector has a fully-distributed com putation architecture which is naturally suited to the hardw are implementation. Its performance is satisfactory in the data detection application. In C hapter 4. the IDD technique is applied to reduce the complexity for the sequence d ata detection when the ID memory is long. First, a generic reduced-state SISO [ChCh99b] is discussed briefly. The concept of self-iteration is also intro duced. A com pact signal model is developed for arbitrary sparse ISI (S-ISI) channel 6 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [ChCh99c]. The corresponding sparse SISO (S-SISO) algorithm s are developed. A list of rules is established to guide and ease the design of a S-SISO based detector. A series of toy and open problems are discussed in C hapter 5. These problems are either directly derived from the previous chapters or made up to gain insight of various problems. Specifically, the complexity of the optim al 2D data detection, the comparison of symbol and sequence detection, the normalization and -V 0 factor in the MSM SISO algorithm s, the distance property of 2D ISI [ChChThAn99] and regular S-ISI [ChCh99c], and the generic likelihood combining problem are discussed. The organization and relationship of various topics in this dissertation is illus trated in Fig. 1.4. The author has made original contributions to the shaded topics. 1. introduction _ _ A 2.2 2.1 interative detection SISO 5.6 j likelihood combining -5.3 MSM Appendix A,B.C 5.1 MLPE. 5.2 bit vs. symbol 3. 2D ISI/AWGX 4. long ID ISI/AWGX A- 3.2 A 3.3 2D concatenated 2D distributed y 4.2 RS-SISO A 4.3 sparse SISO 3.2^3.1 3 ^ 3 .3 minimum distance image halftone Y 5.5 regular S-ISI .. . y 5.4 distance spectrum Figure 1.4: The organization of this dissertation. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. C hapter 2 S oft-In p u t/S oft-O u tp u t A lgorithm s and Iterative D etectio n Like two cornerstones of the iterative data detection, the SISO algorithm, and itera tive detection have enabled a series of successful applications since the invention of the celebrated Turbo code [BeGlTh93. BeGI96]. In this chapter, the SISO algorithm and iterative detection are introduced and their relationship is discussed. 2.1 S oft-In p u t/S oft-O u tp u t A lgorithm s 2.1.1 H ard D ecision and Soft Inform ation The fundamental problem of data detection is to make a reliable decision on certain finite discrete random variable which carries message and is observed with distortions (e.g.. interference, noise). This decision is called a hard decision since it assigns the random variable a unique value in its sample space. For example, based on the observation of a binary random variable a G A = {0, 1}, a hard decision detector (e.g.. a threshold detector) has to choose one point from the discrete sample space A. e.g., a = 1 (see Fig.2.1). Statistically, a = 1 means that it is more reliable to estim ate a as 1 than as 0. However, the hard decision does not provide any quantitative measure of the reliability. Also, its "hardness" implies that it cannot be modified once it is generated. Alternatively, soft information [Fo66] (also called the soft decision) is a reliability measure over the sample space of the investigated random variable. An algorithm which has soft information as its input and output is called a soft-input/soft-output 8 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. P r (a = 1) a = 1 (0 .2 3 .0 .7 7 ) O---- - - A = {0.1} N \ \ P r( a = 0) a = 0 a = 1 o l soft inform ation hard decision Figure 2.1: Comparison of a hard decision and a soft information. algorithm (SISO). Essentially, soft information is either a probability distribution trated in Fig.2.1. In the probability space which is continuous, the soft information is just a point. More informative than a hard decision, the soft inform ation not only tells that the event {a = 1} is more likely than the event {a = 0}, but also how much more likely it is. By thresholding the soft inform ation, a hard decision can be readily obtained if it is necessary. In this example, a = 1 since Pr(a = 1) > Pr(a = 0). The quality of a soft information is usually evaluated by the error probability of the hard decision generated by thresholding it. Actually, the key notion carried with the soft information, also its main advantage over a hard decision, is its "softness'' - i.e., the soft information can be modified. Through a certain ways {e.g.. iteration), a soft information processor {e.g., SISO algorithms) could refine the soft information to make a later hard decision more reliable. Generally speaking, there are two types of soft information, which we refer to as a-posteriori probability (APP) and minimum sequence metric (MSM). The A PP soft inform ation is probability-based while the MSM soft information is metric-based. The definition of a m etric is usually deter mined by the type of noise. In Section 2.1.2.3, a m etric will be defined in the case of an AW’GX. There are only two basic operations applicable to the soft information, i.e.. marginalization and combining. Any algorithm based on soft inform ation involves these two operations. Assume u = f{v\. v2) is a function of iq and v-i, then the soft information about iq (or v2) can be obtained by marginalizing the soft infor mation about u over v2 (or tq) with regard to the function /(•••). i.e.. Pr(tq) = marginalizetM . P r ( u ) . On the other hand, the soft information about u can about a random variable or its variations. An example of soft inform ation is illus- 9 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. be obtained by combining the m arginal information about iq and Vo under the con straint of the function /(-,•). i.e., Pr(u) = combine(t.1 .t,2):u=y(l.ItL .2) ( P r ^ J . P r t e ) } . Fig.2.2 gives a geometric interpretation of these two operations. Although the U = f{v 1 . fo) P(r; c x - P(«) m arginalization P(i-a) O X ? A o com bining P (u) P(Cl) Figure 2.2: M arginalization and combining operations on the soft information. m arginalization and combining may have different appearances in different situa tions. we will see that all the SISO algorithms just use these two basic operations to obtain the desired soft inform ation. 2.1.2 T h e A P P and M S M SISO A lgorithm s for Sim ple FSM s 2.1.2.1 T he Sim ple F inite State M achine The underlying system with memory on which the APP-SISO and MSM-SISO al gorithms operate is a hidden Markov model (HMM) [Ra89]. In general, an HMM consists of a finite state machine (FSM) which is driven by a digital sequence1 a f =' {a*., k = 1. • - •. A'} and a noisy observation process, i.e., s fc+1 = s(sk.ak) = s(qk) (2.1) xk = x(sk, ak) = x(qk) (2.2) Zk = 9(xk. wk) = g(qk, wk) (2.3) where sk is the state of the FSM at time k. x k is the noise-free output of the FSM and zk is the observation. Also s(s,a) is the next-state function, x(s.a) is the ‘We use d£* to denote the vector (sequence) {dk • 10 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. output function, and g(x, w) is the observation process. A transition is defined as (jk =f (sk-Q-k)- These quantities are illustrated in Fig.2.3. In this dissertation, we Figure 2.3: Param eters defined for a generic finite state machine. always assume that the FSM is homogeneous and tim e-invariant, and the noise is an AWGX. It is conceptually straightforward to extend all the following arguments to inhomogeneous, time-varying FSMs and/or other types of noise as long as the necessary knowledge about the HMM is available. As a special case of HMM. a simple FSM with memory L (e.g., an (L + I)-tap ISI channel) is mostly used in this dissertation for the simplicity of presentation. For a sim ple FSM. the state Sk =f a £ l l L is just a segment of its input sequence. The input symbol ak is independently distributed over a finite alphabet A = {A0. -4 i,. .. which has |M| elements. We assume that the value of a* is known when k £ [1, A'], usually assumed to be .40. Note that the informative observations should be z f ~ rL. In a simple FSM. a transition is also uniquely determ ined by a pair of states, i.e.. qk = (.$k-Sk~i)- These quantities for a simple FSM are shown in Fig.2.4. Additionally, some state sets Qk Sfc+l Sk • • • • O-k-L Figure 2.4: A simple finite state machine. 11 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. associated with a simple FSM are defined for future use: T (j) = {/ : (5, — > Sj) is an allowable forward transition} (2.4) B(i) = {j : (Si < — Sj) is an allowable backward transition} (2-5) Cd(m) = {j : Sk+i = Sj is consistent with ak-d = Am}. d = 0. - • -. L — 1 (2.6) where 5, and S} are elements of state value set S = (5 0. Si, • • •, S|sj_i} of which the cardinality is |S| = \A\L. Due to the memory of an FSM, its current output depends on the state history. The Markovian property of an FSM says that given the current state, the future outputs are independent, from the past state history, i.e.. Pr(z£+1|z?,s{+l) = Pr(zJ+1|zf, »*,.,) (2.7) This is also termed as the folding property in the detection theory since it is precisely the property required to fold an exhaustive tree search into a trellis search [Ch98]. It is also the property on which the YA relies to make pairwise local decisions on paths entering a given state without regard to the future observations. Furthermore, when the state of an FSM at a certain time is given, the equation P r ( z f +£|sfc+1) = P r(zf|s* +1) P r ( z ^ i£,|sfc+1) (2.8) holds. It is referred as the time splitting property since the given state makes the past and future observations independent. Along with the Bayesian rule, the folding and tim e-splitting properties enable the APP/M SM -SISO algorithm in the following sections. It should be noted that in principle, only one combining operation and one marginalization operation are needed to com pute the desired soft information, e.g.. K P r(afc ,z * + t) = Y 1 P r ( a f . z ^ /') = ^ ( P r ( z ^ L|a f ) P r(a ,)) (2.9) »» A " a A * I — 1 a l *a Jc -f-1 a l fc-r 1 However, the nice properties of the simple FSM enable the same task to be accom plished recursively and therefore simplify the com putation dramatically. 12 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2.1.2.2 T he A P P -SISO A lgorithm for a Sim ple F SM In the APP-SISO algorithm for a simple FSM, given the observation segment z ^ . we define the following probability-based quantities: a*, U) = Pr(sfc+i = Sj, zk kl) (2.10) ^ 1 , ( 0 = P r (z tu \s k+l = St) (2.11) 7k(iJ ) = P r(sfc+i = Sj, zk\sk = Si) (2.12) = Pr(zk\qk = (Si, Sj)) P r(ak = A(i.j)) (2.13) where A(i,j) is the value of the symbol determ ining the forward transition (S', — » Sj). We denote the probability-based soft information about d as P(d). The soft inputs of the APP-SISO are Pi(a;t) = Pr(ajt) and P,(xk) = Pr(c^-|j:*;). which are the a priori knowledge about ak and xk. Using the folding property, both ctki(i) and ,3kl.i(j) can be com puted recursively via: <*£,0') = Z (2-1-1 ) ‘€X(j) 3 & ,(0 = Z (2-15) j€B(i) The forward recursion for a T s is initialized by a£[- 1(i) = Pr(.s/t[ = S,) and the backward recursion for J's by 3k^ t(j) = 1 for arbitrary j because 3g(i) = Pv(zk2\.sk2 = Si) = ] T 7k2(iJ ) ' 1 (2-16) jes(i) Once both (j) and > 3k are available, the completion step is executed to output joint probabilities about ak and x k: P r K = .4 ,„ ,* * ;)= Z (2-17) « 6 C o ( m ) Pr(x* = x (S ,.S y ).z £ ) = a^~l{i)yk(i,j)3^+l(i) (2.18) 13 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. W hen the APP-SISO operates on an isolated system or no further iteration is re quired. these joint probabilities are the final soft output. Specifically, if the under lying FSM is an isolated system, the soft inform ation P r(afc .z ^ ) is optim al in the sense th at by thresholding it, we get the maximum likelihood bit(symbol) estimate (MLBE). On the other hand, when the APP-SISO algorithm is put in an iterative detector, it should output the so-called "extrinsic" soft information2 [BeGlTh93] which is defined as: P0(a,) d =f P r(a* ,zg )/P s(a* ) and P0(xk) ■ ^ Pr(x*. z£)/P i(a:fc ) (2.19) and the corresponding completion step becomes Po(a* = A m) = < l ( 0 p i(** = -r(St.S ,) ) j £ .1(J0 (2-20) PQ (xk = x{Si. Sj)) = a * - l(i)P,(a* = A{i,j))3**+l(j) (2-21) We call this SISO algorithm as the APP-SISO for a simple FSM. As shown in Fig.2.5. it consists of the forward recursion (2.14), the backward recursion for (2.15) and the forward recursion > < - ’ --------►a*"1------- ► < ------ > < ~ 1------> ;. . ; P k - 2) P(af c _!) ^ P ( a O , P ( u ^ i) • < — 3 t i< 3 ? < --------.3&L<------- 3tU< backward recursion Figure 2.5: Com putational architecture of the APP-SISO for a simple FSM. completion step (2.20)-(2.21), in which the two basic operations on the soft infor m ation - m arginalization and combining - correspond to the summation and mul tiplication. respectively. In general this correspondence is true for any probability- based SISO algorithm , which therefore is also called the sum-product algorithm ■This concept will be discussed in detail later. 14 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. {e.g.. [\Vi96]). It is easy to see that the com putational complexity of this algorithm is mainly determ ined by the number of states of the underlying FSM which grows exponentially with the length of memory L. It is notable th at it is conceptually straightforward to generalize this APP-SISO to some more general FSMs. e.g.. the embedded FSM for a recursive systematic convolutional code or a TCM code (with parallel transitions). 2.1.2.3 The M SM SISO algorithm for a sim ple FSM Define the metric-based soft information as the negative-log of its probability-based counterpart, i.e., Xl(d) = —cx ln(P(d)) -j- c2 where cx > 0 and c2 are constants that may be selected to simplify the calculation. Specifically, in the case of AWGN with variance a1, let cx = 2cr and c2 = — cL ln(\/27r<T). we obtain M i(xt) = Ic* — xk\2 which is the Euclidean distance, a m etric in the observation space. This is why M(-) is called the metric-based soft information. Let the transition m etric Xk{i,j) =f Mi(j'*) + Mi(ofc) be the metric-based counterpart of 7fc (L j). where xk = r ( S t. Sj) and ak = A(i.j). For a simple FSM. a (valid) sequence of states uniquely determines a sequence of transitions. Then, the metric of a state sequence is defined as the sum of metric of the corresponding transitions. Also, we can define two minimum sequence metrics as It is easy to verify that Aj^O) + Bkl_l{i) is the minimum sequence metric of the state sequence which passes the state sk+1 = Sx. According to their definitions, both (2.22) and (2.23) can be computed recursively via: k mm > k~l .. _< r ■ • k i ~ n_k (2 .22 ) s, n=k+i A ; 2 Y . A„(u. c) (2.23) A*, O') = min [Ak k l(i) + Xk{i.j)\ 'e^O) Bfctii) = m i n [ ^ 'I 20 ) + X k+l(i,j)] jeB(i) (2.25) (2.24) 15 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The forward recursion for A's is initialized by A kl~l(i) = = Si) and the backward recursion for £Ts by B k-+l{i) = 0. Once -4£ (z) and B kl_l{i) are available, the com pletion step can com pute the following soft information: M(ak = .4m,z g ) = . min (.4 ^ (0 + (2.26) i € C o ( m ) M(x* = x (S „ S ,-).z £ ) = .4‘r '( i) + At (i. j ) + B f ii( j) (2-27) if the underlying FSM is isolated or no further iteration is required. Specifically, when the underlying FSM is an isolated system, the soft information M (a*.- = Am,z k2) precisely presents the minim um metric of a sequence passing a* = .4m. This is how the term "minimum sequence metric" comes about. By thresholding M(ak = .4rn.z ^ ) . we obtain exactly the maximum likelihood sequence estimation (MLSE) as the VA does. Similarly, the "extrinsic" m etric-based soft information: M0(afc ) ^ f M (afc , z ^ ) - M i ( a fc) and M0(r t ) ^ M(x*. z g ) - Ms(z*) (2.28) is preferred when the iteration is required and the corresponding completion step becomes: Mo (a* = -4m) = min [ A ^ i i ) + Mj(x* = x(Si,Sj)) + B?+lU)] (2-29) (S,.Sj):.4(i.j)=.4m M0(xk =x{Si,Sj)) = A k~l{i) + Mj(af c = A(i. j)) + B ^ ( j ) (2.30) This SISO algorithm is called the MSM-SISO for a simple FSM. It has almost the same com putational structure as the APP-SISO (Fig.2.5). Specifically, both the forward and backward recursions in the MSM-SISO have the same structure as the forward recursion in the VA, which uses the well-known add-comparison-select (ACS) operation. Unlike the APP-SISO, the m arginalization and combining opera tions in the MSM-SISO correspond to the minimization and summ ation operations, respectively. Therefore, in some works {e.g., [Wi96]), the metric-based SISO is also called the min-sum SISO algorithm . Similarly, the complexity of MSM-SISO is pro portional to the number of state. The generalization on the underlying FSM for the APP-SISO can be applied to the MSM-SISO in the same manner. 16 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2.1.2.4 A P P /M S M -S IS O s and a Translation Rule It is very im portant to observe th at the APP-SISO and the MSM-SISO have exactly the same com putational architecture: a forward recursion and a backward recursion over the same trellis and a com pletion step. Therefore, we can view these two algo rithm s as two versions of the SISO for a simple FSM. W ith little effort, a translation rule can be established between the A PP and MSM versions of the SISO algorithm for a simple FSM. As listed in Table 2.1. this translation procedure contains three steps: 1. Replace the APP (MSM) quantities with their counterparts: 2. Replace the m ultiplication (addition) operations with their counterparts: 3. Replace the sum m ation (minimization) operations with their counterparts. In fact, this translation rule is so general th at it can be used to derive the APP(M SM ) version of any SISO algorithm from its counterpart. Computationally, the MSM version of a SISO is much less complex than its A PP counterpart because it uses simpler operations. translation A PP version < = > MSM version soft information A PP quantities MSM quantities combining m ultiplication addition m arginalization summ ation « = > ■ minimization Table 2.1: Translation rule of the A PP and MSM versions of a SISO algorithm. It is notable that in some of the literature, a so-called log-APP version [RoViHo95. BeMoDiPo98] of the SISO algorithm is given. The log-APP SISO can be directly obtained through replacing the min(-) function in the MSM-APP by the min*(-) function which is defined recursively as min*(f1 ? t2, • • •. tk) = min*(fi, m in '(f2, • • •, h-)) min*(fi, t2) = m in(fl. t2) — ln (l -I- e x p ( -|f l - t21)) (2.31) It can be verified that the log-APP-SISO is ju st a variation of the APP-SISO which operates in the negative-log domain. Note th at the MSM-SISO can be obtained 17 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. from the log-APP-SISO b\- replacing min*(-) with min(-). W hen the signal-to-noise ratio (SXR) is high, min(-) is a good approximation of min*(-). In some references {e.g.. [LiVuSa95. RoViHo95, Wi96, BeMoDiPo98]), the MSM-SISO is viewed as an approximation of the APP-SISO and is therefore "sub-optim al." However the MSM- SISO has a rigorous m athem atical interpretation and yields the (optimal) MLSE. For an isolated FSM. we prefer to think that both the APP-SISO and the MSM-SISO are optimal but in different senses [HaCoRi82]. W hen the underlying FSM is just a subsystem, neither the APP-SISO nor the MSM-SISO is optim al when iteration is used. In the terms of sym bol(bit)-error-rate (BER) which is the widely-used perfor mance measure, the APP-SISO does perform better than the MSM-SISO (this is an empirical observation). However, it has been shown numerically that both versions have only a marginal performance difference {e.g., [LiVuSa95. ChCh98b. AnCh98]) which is negligible in most applications considered in this dissertation. Therefore, the MSM-SISO should be preferred to the APP-SISO due to its com putational simplicity. It is notable th at intuitively the Pi(a*..) and Mi(a*) correspond to the average likeli hood (AL) and generalized likelihood (GL) of ak [Va68] when the nuisance param e ter is 9 = i.e.. Pj(afc ) ~ Y,o P r[a f \ z£ ] and Mj(a*) ~ max* Pr[a{\ z£]. The MSM quantities cannot be obtained from the A P P quantities and vice versa. 2.1.3 C ategorization and H istory o f SISO A lgorithm s As discussed in the previous section, based on the type of soft information used, a SISO algorithm can have three different versions, i.e.. APP. log-APP and MSM. From another viewpoint, the range of observations used is an im portant factor for determining the quality of the soft output and the complexity of the algorithm. Specifically, for the input symbol a^ at time k, the fixed-interval (FI) SISO uses all the informative observations {i.e., z f ~L) to com pute the soft output about it. the fixed-lag (FL) SISO uses z \ ^ D where D is the sm oothing lag. and the (symmetric) fixed-window (FW) SISO uses z£ l£ where 2D is the window size. These three cases are shown in Fig.2.6. By assigning appropriate values to the parameter A q and k-> in Section 2.1.2, we can readily obtain the FI/FL /FW -A PP/M SM -SISO algorithm for a simple FSM. The larger the observation range used is, the better the qual ity of the soft output. W hen D is increased, the performance of the FL/FW -SISO 18 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. FI-SISO h FL-SISO I - FW -SISO forward backward recursion t recursion Qi 1 k - D Q1 1 k ~ D . k — D J fc -l index k - u k ~ u K Figure 2.6: Information collection ranges of FI/FL /FW -SISO algorithms. will approach th at of the FI-SISO. Actually, as in the VA with a decision delay D [HeJaTl], the difference in performance among these SISOs is negligible after D is greater than 5L ~ 7L [LiYuSa95. AnCh98]. Although having sim ilar performance, these three types of SISO have different com putational complexity and paralleliz- abiiity. From the Table 2.2. it is easy to see that the FI-SISO has the lowest average SISO FI FL FW FR 1 1 D BR 1 D D RP I< I< D RS 2 A' K + D I< + D DP(fc) I< D (k < D ). k (k > D ) D DS(fc) 2(I< - k) D D Table 2.2: Complexity comparison of FI/FL /FW -SISO algorithms. Xote: FR /B R is the average number of forward/backward recursions used to yield the soft infor mation about ak. R P /R S is the number of recursions/tim e-slots used to yield all the soft information when the whole soft input set is available in parallel/serial manner and the com putation ability is infinite. DP/DS(Ar) is the number of recursion/tim e slots used to yield the soft information about ak when the whole soft input set is available in parallel/serial manner and the com putation ability is infinite. com putational complexity. However, since it cannot provide soft outputs until the whole soft input set is available, the FI-SISO has the highest time complexity when the input is in serial manner. Therefore, a FI-SISO is suitable to applications in which the input is obtained in a parallel m anner, e.g., the Turbo codes [BeG196]. 19 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. serial concatenated convolutional codes (SCCC) [BeDiMoPo98] and page-oriented storage system s [ChCh98, ChChOrCh98]. The average com putational complexity of the FL-SISO is D times higher than that of the FI-SISO. However, its time complex ity is lower when the input m anner is in serial. Therefore, the FL-SISO is suitable for applications where the decision delay is critical. Actually, the time complexity of FL-SISO can be reduced by using different com puting architecture as in [LiVuSa9o] or in Appendix A at a cost of heavier com putation. The FW-SISO has the highest average com putational complexity. But it has a distributed com putational archi tecture. Therefore, it is suitable to applications having a parallel input interface. Compared to the FI-SISO. the FW-SISO has a more efficient distributed structure but a heavier com putation. In Appendix B. some complexity saving techniques are presented for various SISO algorithm s without suffering any performance degrada tion. Over the last 40 years, the history of he SISO algorithm has been rich. To the au th o r’s best knowledge, the SISO is first discussed in a m athem atical paper [StGOj in 1960. Then in the engineering literature, the unpublished Baum-Welch algorithm [Ra89] includes a FI-APP-SISO for a general HMM. Chang and Hancock [ChHa66] developed the same FI-APP-SISO for a linear ISI channel. Bahl et al. [BaCo.JeRa74], developed a symbol FI-APP-SISO for the specific FSM of a linear code which is often called the B C JR algorithm. The same algorithm also appeared in the Ph.D . dissertation by McAdam [Mc74]. The FI-SISO algorithm for a simple FSM is later generalized to accommodate non-simple FSM structures in [BeG196] for recursive convolutional codes and in [BeMoDiPo96] for FSMs with parallel transi tions. Numerous authors have presented the FI-MSM-SISO based on the high-SXR approxim ation (e.g., [RoViHo95, Wi96, BeMoDiPo96]). The FL-APP-SISO and its variations were developed first by Abend and Fritchm an [AbFr70], and later refined by Lee [Le74] as well as by Li, Vucetic and Sato [LiYuSa95] with these two being essentially identical. It was developed further by Benedetto, et al. [BeMoDiPo96] and then by Chugg and Chen [ChCh98b]. The structure of Lee and LYS (L2VS- SISO) and FL-SISO have a complexity growing linearly in D, whereas this growth is exponential in Abend Fritchm anT s structure. Unlike the bi-directional structure of the FL-SISO developed, the L2VS-SISO only requires forward recursions. Although 20 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. the L2YS-SISO and FL-SISO have different com putational architectures, it can be shown in Appendix C th at they are equivalent. 2.1.4 C ode N etw orks and SISO A lgorithm s In the previous sections, the SISO algorithm is developed and discussed for an iso lated FSM. However, usually the underlying FSM is not isolated, but is ju st one of many memory sources in a system. The concatenated code [F066] is such a system. By cascading two or more relatively sim ple constituent codes, the concatenate code is proposed to achieve a large coding gain. As shown in [F066], in order to obtain this gain, the output of the inner decoder should be the likelihood about the inner code word conditioned on the observations and the code structure. However, the expense of finding this optim al (maximum likelihood) soft information is prohibitive high since the code is so large. Fortunately, the concatenated code is practically enabled with a suboptim al but effective decoding scheme: iterative decoding. By ignoring other constituent codes, a decoder is built for each constituent code and the soft information is exchanged through iterations among these decoders to make the final decision reliable. Actually, the construction of the Turbo code in [BeGlTh93] and the low density parity check codes in [Ga63. Ta81] are based on the same decoding strategy. In [BeMoDiPo98], a code network is defined as a generalization of the concate nated code. Besides of the FSM "encoder' 3 which defines the constituent codes, other standard building blocks, i.e.. the interleaver, mapper, parallel/serial/parallel and broadcaster (see left side of Fig.2.7) are defined to assemble the constituent codes together. Consequently, the construction of a code network can be much more general than th at in [F066]. For example, it includes the Turbo code in [BeGlTh93] which is made up of a broadcaster, m ultiple RSCs and interleavers, the SCCC in [BeDiMoPo98] which is in turn made up of m ultiple convolutional codes and in terleavers, and even a TC M /interleaver/ISI channel. As long as a system can be constructed by these standard building blocks, no m atter how complicated it is. an iterative decoding scheme can be readily established by using the corresponding 3The term “encoder” is used for the effect imposed by an FSM. intentionally or not. Therefore, the decoding problem can be viewed as a special case of the generic detection problem. 21 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. a >» FSM X > - P(a)^ > ■ SISO ■ < *P (z) FSM Encoder a X P (a )^ » ■ 7 T _*P (z) _— I ji ■ < - Interleaver Q h t ^ /(•) £ P (a D / - ( • ) ■ < P(z?) Mapper ■M P arallel / S erial/P arallel P (a) •P(a) P(z?) Broadcaster Figure 2.7: Building blocks for the code network and the iterative decoder. decoder building blocks based on the network structure. As shown in Fig.2.7. the SISO algorithm is just a key building block of the iterative decoder corresponding to the FSM "encoder". Ignoring the internal structure, it is a four-port device which can refine the soft information about the input and output of the underlying FSM. Additionally, a soft-input/soft-output device can be developed for each code net work building blocks (please refer to [BeMoDiPo98]). Each decoder building block is a device which is able to yield the desired "extrinsic" soft information by apply ing the marginalization and combining operations. Actually, these devices can be treated as SISO algorithm s in the general sense to which the SISO algorithm for the simple FSM is just a special case. It should be noted th at due to the lack of structure of underlying systems, some of such SISO algorithms (e.g., soft mapper [BeMoDiPo98]) are fairly complex. An iterative decoding scheme is then built by re placing the building blocks in the given code network by the corresponding decoder 22 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. building blocks and connecting them properly. It has been shown numerically that the iterative decoder obtained by this way is effective. 2.1.5 B ayesian N etw orks, E xtrinsic Inform ation and SISO A lgorithm s More generally, any belief propagation algorithm (BPA) built on a Bayesian network can be viewed as a SISO algorithm [Pe86. Pe93]. A Bayesian network [PeS6] (also called a belief network) is a directed acyclic graph in which the nodes represent the random variables under investigation, and the directed edges represent the depen dence between the nodes. Fig.2.8 shows two Bayesian networks. The belief about a “ 0 Pr(“) P r(r|u ) cvcles / non-cvcle-free cycle-free Figure 2.8: A cycle-free and non-cycle-free Bayesian networks. node is defined as the soft inform ation about it. and the conditional belief (depen dence) about an edge as the conditional probability (or m etric) of its two nodes. A BPA is an algorithm which can yield the belief on the nodes by fusing and propagat ing (i.e.. marginalizing and combining) the belief from the known nodes and edges. The information flow paths in the BPA corresponding to the Bayesian networks in Fig.2.8 are illustrated in Fig.2.9. In [I\sFr98], the correspondences am ong the Markov random field [GeGe84]. the Tanner graph [Ta81] and the Bayesian network are es tablished. Consequently, it is easy to see (as in e.g., [Wi96, MaXe96, McMaCh98]) that all the SISO algorithms discussed above, even the EM algorithm [DeLaRuTT]. 23 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. can be treated as special cases of the BPA. Also, we will find out in the later chap ters th at some general SISO algorithms can be readily derived in the framework of Bayesian networks. Si soft inform ation flow Si tO & * o optim al BPA ad hoc BPA Figure 2.9: Information flow paths in the BPA. W hen a Bayesian network is cycle-free4, it is easy to show th at the resulting fusion and propagation can be accomplished. It should be noted th at in this case no iteration is required because the BPA can utilize the evidence once and only once in the optim al way. For example, by introducing nodes representing transitions, a cycle- free Bayesian network model can be established for the FSM defined in (2.3). As long as the FSM is isolated, the resulting BPA is equivalent to the APP/M SM -SISO developed in Section 2.1.2 which is optimal. However, once the Bayesian network {e.g.. the right one in Fig.2.8) is not cycle-free, no BPA can be obtained in the rigorously m athem atical m anner as in [Pe86]. Several approaches [Pe86. BeGe96] {e.g.. collapsing the nodes in the cycle into a single node) have been proposed to resolve this problem optimally. Unfortunately, in general, these approaches are very complicated. Alternatively, an ad hoc BPA can be derived by ignoring the cycles in the Bayesian network. Such an ad hoc BPA is illustrated in Fig.2.9 for the non-cycle-free Bayesian network in Fig.2.8. The belief yielded by this BPA is 4 A cycle-free graph has no undirected cycle in it. It is also called the singly connected graph in BPA is optim al [Pe86, Wi96] given the soft input is correct because perfect belief [Pe86j. 24 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. not optimal in any sense because the approxim ated belief fusion and propagation are used. Nevertheless, the nature of the BPA allows the multiple runs of belief fusing and propagation over the Bayesian network. We refer to this approach as an iterative BPA. Although in each iteration, the BPA is suboptim al or even poor, the iteration can fuse the belief in a more effective way by exploiting the network structure that is om itted in the first place. Consequent!}*, a more reliable belief can be obtained through iterations. However, due to the use of iterations, the same evidence may be accounted for several times. Like undesired positive feedback in a circuit, such reuse of the same evidence would degrade the performance significantly [Wi96]. Therefore, any direct feedback of the belief should be excluded from the iterative BPA. This results in the use of the "extrinsic" information, which is defined as the residue belief after removing the a-priori belief from the a-posteriori belief. Basically, the "extrinsic" inform ation is a generalization of the mutual information in the information theory [CoTh91]. By deducting the a-priori information, the "extrinsic" information keeps "only" (not rigorously) the "new" information extracted through the belief propagation (see Fig.2.10). W hen the "extrinsic" information is used in Figure 2.10: An information theory view of the “extrinsic" information. the next iteration, the reuse of the a-priori belief is explicitly avoided. It has been shown that the use of "extrinsic" information is crucial for the success of the iterative data detection {e.g., [BeG196]). m utual inform ation a-priori inform ation' a-posterion inform ation extrinsic inform ation 25 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2.2 Iterative D etection 2.2.1 P rinciple Both the iterative decoding scheme and the iterative BPA mentioned above are just examples of a generic approach: iterative detection. A popular idea used in solving a hard problem is to break it into a group of dependent simple sub-problems. Solving each sub-problem locally and exchanging the local information with other dependent sub-problems could efficiently solve the underlying hard problem in a sub-optimal (usually near-optimal) m anner (see Fig.2.11). When applied to the field of data detection, this approach is called the iterative data detection (IDD). also called Turbo detection because of its key role in the success of Turbo codes [BeGlTh93]. Due to the use of iterations, the soft information becomes the natural candidate for the exchangeable information. Consequently, a SISO algorithm is needed for each sub-problem. When there is no need for the iterative detection, the SISO algorithm can usually be replaced by a simpler hard-decision approach, e.g.. the MSM-SISO can be replaced by the VA. Therefore, it is fair to say th at the SISO algorithm and the iterative detection strongly rely on each other to be useful. It should be noted that in the iterative detection scheme, the APP/M SM -SISO algorithm is applied in a suboptimal manner since the independence assumption about the input symbol usually fails for a local FSM (e.g., the inner code in an SCCC). local SISOs sort information , \ exchange Figure 2.11: A subsystem network and the corresponding iterative detection scheme. 26 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The IDD technique can be used to solve the hard detection problem arising from the real-world transm ission/storage systems. For exam ple, the generic multiuser detection problem [Ye98] can be very hard. By treating the detection for a single user as a sub-problem, the iterative detection scheme can solve this hard problem in a near-capacity m anner [\Io98]. It is also possible to design a hard problem to which the iterative detection is readily applicable. For example, it is well known that the longer an error-correction code is, the better its performance is and unfor tunately. the more com plicated its optim al decoding scheme is as well. Although the Turbo code [BeG196] is such a long code [BeMo96], it is designed in the way that its two constituent codes can be readily decoded by A PP/M SM -SISO algorithms. Then iterations help to exchange the soft information between two sub-problems. Although the Turbo decoding scheme is sub-optimal, the Turbo code still has a near Shannon-limit performance. Although iterative detection has been used as a powerful tool in many applica tions since the invention of Turbo codes, a complete understanding of its success has not been obtained yet. In [\ViLoKo95. Wi96]. W iberg showed th at when the underlying code has a finite, cycle-free Tanner graph, the iterative decoding scheme converges. Later in [Mo97. \IoGu98]. Moher proved the convergence of the itera tive cross-entropy m inim ization algorithm , and showed th a t the iterative detection scheme is just a sub-optim al im plem entation of that algorithm . Recently in [Gr99], G rant showed that the general marginalized iterative relative entropy algorithm , which is closely related to the iterative detection scheme, has fixed points. The conditions for its monotone convergence are also given. Also in [Ri99], a geometric interpretation of the iterative decoding algorithm is presented which clearly indi cates the relationship between turbo-decoding and m axim um likelihood decoding. It seems th at more works are needed for a better view of the iterative detection scheme. 2.2.2 Iterative D etectio n for M ultiple D im en sion al S ystem s Usually, a detection problem associated with a multiple dimensional system, e.g., the page-oriented memory [ChChNe99], is hard. A natu ral way to view a m ulti ple dimensional system is to treat each dimension as a subsystem. Consequently. 27 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. the original multiple dimensional system becomes a network of one dimensional subsystems. One can readily apply the IDD technique to such a network. Further more. the embedded system in some d ata detection problems can be modeled as a multi-dimensional system as well. For example, the adjacent channel interference can be modeled as a two-dimensional ISI. Because of its excellent performance, the iterative detection has been used in various multi-dimensional data detection prob lems. such as near-capacity m ulti-user detection [Mo98]. near-optim al page-oriented d ata detection [ChCh98, ChChXe98] and image halftoning [ChChOrCh9S]. In next chapter, iterative detection schemes are developed for the problem embedded in a two-dimensional system. 2-2.3 Iterative D etectio n for D ecom p osed S ystem s Sometimes, a detection problem in a ID system could be very hard as well. For example, for a TC M /interleaver/ISI/A W G X channel [AnCh98], the VA built on the overall trellis is just too complicated to be implemented. In many cases, this one dimensional system can be decomposed to form a network of sim pler sub-systems to which the iterative detection technique is readily applicable. There may exist several different decomposition schemes for the same system. The complexity and performance of the corresponding IDD scheme are strongly dependent on the way of decomposition. It is always preferred to decompose a system in a way so that the resulting sub-system network will be as simple as possible. In C hapter 4. certain decomposition schemes for a long/sparse ISI are presented and corresponding IDD schemes are developed. 28 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. C hapter 3 Iterative Tw o D im ensional D ata D etectio n 3.1 B asic A ssum ptions and Signal M od el A two dimensional (2D) d ata detection problem aims at finding am ong a discrete set of choices, an estim ate of a data m atrix1 A =f i) such that a given quality measure over the whole m atrix is optimized. This d ata m atrix can only be observed through some non-ideal procedure suffering linear interference and distortion. Specifically, the signal model is described as: =(i-j) = g{x(i.j). w(i.j)) (3.1) Lc Lr x(i-j) = * a(i- j) d = ~ L J ~ m ) (3-2) I — — Lf 7 7 1 — “ Lr where g(x.ix) is the distorted observation process, w(i,j) is the distortion, and x(i,j) is the output of the linear interference which is modeled as a 2D ISI defined by { f -Lr)' 3 5 5 1 1 1 1 1 6 th 3 1 th*? distortion w(i,j) is additive. It is easy to adapt the algorithm developed in this chapter to other signal models arising from real-world systems [ChChXe99]. Given a page estim ate A, we define a pixel cost (merit) function X(i.j) =f A(z(i. j). x(i, j)) to measure the quality of the estimate, where x(i.j)) = f(i.j) * a(i,j). Similarly, this quality measure function can be either probability-based or metric-based. Specifically, for a 2D ISI/AW GX channel and the IID d ata m atrix, we can define a probability-based merit function A(i.j) =f ‘The capital, bold-faced letter U is used to denote the matrix {* * (* ’../’)}(«._/)- 29 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Pr(c(L_/)|ir(L j)) at pixel (i.j). Then the overall m erit of the page estim ate A is just P r(Z |A ) = ri(ij) M hj). When a page estim ate is obtained through A = arg m ax P r(Z |A ), (3.3) A it is called the maximum likelihood page e.s£imaie(MLPE) which is optimal in the sense that the probability of page error is minimized. Alternatively, we can define a metric-based cost function A (i.j) = = |c(L j) — x{i. j )|2 with the corresponding overall cost as E(I J ) lc(z.j’) — x(/._/)|2. Then the MLPE can be obtained through A = a x g m m ^2 \z(i.j) - x ( i . j ) j2. (3.4) {i.j) This is also called the digital least metric (DLM) [ChChOrCh98] problem, the ob jective of which is to find the closest (in the sense of Euclidean distance) restricted lattice point in a 7£‘ V lX :' 2 space [Ya97]. A straightforward im plem entation of an MLPE detector is to build a look-up table (LUT) storing the cost of all possible input pages. However, the number of entries in this LUT is |A| N ‘A- which makes this conceptually simple implementation prohibitively expensive even for a binary data page with medium size. Another approach to realize the MLPE detection, illustrated in Figure 3.1. is to run the YA by treating each row or column in A as N N T state j T state j -r 2 state j -r 1 Figure 3.1: An implementation of the MLPE detector using the YA. a single vector symbol. Again, the complexity of this im plem entation grows expo nentially with the size of pages. Actually, it will be shown in Section 5.1 th at in 30 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. general this MLPE detection problem is N P-hard [GaJo79] since the 2D index set lacks of a natural order. If one problem is proven to be NP-hard. one should not focus the research on the quest for a general, efficient algorithm , but on the design of good special-case, average-case. and approxim ation algorithms. Hence, to find an approxim ate solution is much more promising than to find an optimal and efficient approach for the 2D data detection problem. As mentioned in Section 2.2.2. the IDD technique can be a good solution for this m ultiple dimensional problem. 3.2 A n Iterative C on caten ated D etector 3.2.1 A C on catenated C hannel M o d el A natural attem pt to decompose the 2D ISI in (3.2) is to check whether the coefficient m atrix {f(i.j)} is separable, i.e., the outer product of two ID ISIs. However, it is easy to show that in most cases the 2D ISI is inseparable. Finding a generic way to decompose 2D ISI/AW GN channels is necessary. A simple modeling trick allows the 2D ISI to be represented as a concatenation of two ID ISIs connected by a block interleaver. Define the row vector as an inner vector symbol. The 2D convolution operation in (3.2) may then be reformulated as where f„ , = [/(m . Lr) f(m . Lr — 1) • • • f(m , —L r)] and (•)' denotes the m atrix trans pose operation. This serially concatenated signal model for the 2D ISI is illustrated in Fig.3.2(a). An example of above mapping procedure for a 3 x 3 2D ISI and a block diagram of this model are also illustrated in Fig.3.2(b) and (c), respectively. In this model, the outer FSM simply works as a row-wise scalar-to-vector m apper with a memory of 2Lr, which preserves information perfectly. This FSM m aps the original data m atrix A into another m atrix of which the entries are the vector sym bols defined in (3.5). It is notable th at when the input m atrix A is independent, Vj(z) = [a(f, j - L r) a(i.j - Lr -H i) - • • a(i.j + Lr) ] (3.5) (3.6) 31 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. . . . . a(l.j) ------>^Row ID FSM J------> v ( l.j) .... a(2.j) .... ------>^Row ID FSM J------> v (2.j) • • - T • ^ •S' - - n . • .- .T • • V V * V ----a ( N .j ) ----- ------>{Row ID FSM ------> v (.Y .j) Example (3 x 3) ~o 6, cF o; o, 1 9 . o binary (b) > O > O > O V X (i.j) (a) 6-arv 1 3 s c n U . u-'(i-j) S CO u. a • ■ o J CO U . x(i.j) =u.j) Block Interleaver Outer FSM Inner FSM Figure 3.2: (a) A serially concatenated signal model (col-row version), (b) mapping procedure for a 3 x 3 2D ISI channel, (c) the block diagram . the resulting vector symbols down a given column are also independent. The inner FSM defined in (3.6) is actually a generalized ID ISI with vector coefficients fm and vector input symbols Vj (i). Since the operation of the inner FSM is column-wise, an interleaving operation is implicit. Being precisely equivalent to the 2D ISI described in (3.2). this serially concatenated signal model can be viewed as a code network to which the IDD scheme is readily applicable. Note th a t we refer the structure illus trated in Fig.3.2 as the "col-row” version of the concatenated model. The row-col version of this model can be readily obtained by defining the inner vector symbol as a column vector in the similar m anner as (3.5). 3.2.2 2D SISO A lgorithm s and Iterative D etecto rs By using the SISO algorithm for a simple FSM, we can readily build an iterative concatenated detector mirroring the concatenated model in Fig.3.2(c) structurally. 32 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Illustrated in Fig.3.3(a). this detector consists of two types of processors: the column Co. SISO Row SISO Row SISO Col. SISO Col. SISO Row SISO Sl.(ati.j)) = R[a(i.j J ] S / , ( * ' ( i.j)) = R[:(i.j>\s{i.j)\ Soft Information on v(i.j) Final Iteration (a) SI,(a(i.j row SISO COL SISO ( b ) Figure 3.3: (a) A simple 2D SISO: the iterative concatenated detector (col-row version), (b) the block diagram of this simple 2D SISO. SISO and the row SISO. Because they are built for different FSMs. they have different parameters which are listed in Table 3.1. In a 2D system, the observation can be obtained page by page. So we can use either the FI-SISO or the FW-SISO. The procedure of an iterative concatenated detection algorithm (col-row version) based on the APP-SISO is given below: After receiving the observation page Z perform: 1. Compute ancl store P\(x(i.j)) = Pr(^(L j)|x (i. j)). 2. Set P\(a(i.j)) and P\°\vj(i)) according to the a priori information about a(hj)- 3. Set the iteration counter 1 = 0 and the maximum iteration number Im. 33 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Algorithm No. of SISO's Transition Symbol Definition Size Definition Size Column SISO N o s{Fj) |_ 4 |(2£r + l)x(2r.c-r-l) Vj(i) Row SISO A'i Vj(i) L4j(2Lr*L) a(i.j) 1 -4 1 Table 3.1: The param eters of column SISOs and row SISOs in the iterative concate nated detection algorithm (col-row version). Note: s(i.j) =f {vj(? — / ) ■ 4. Column SISO: compute and store P ^ ( vj(0 ) for each column. 5. Row SISO: Using (vj(i)) from step 4 as the soft input about the output of the outer FSM. compute and store P;f~l> (vj(i)) =f P0(v_,(i)) for each row. If I = Im. compute and store PQ (a(i,j)). 6. 1 = 1 + 1. If I < I,n, go to step 4: otherwise thresholding P0(a{i.j)) to make hard decisions a(i,j). Note that usually the m axim um iteration number Im can be estim ated when the channel is known. An alternative stopping condition is to let row SISOs calcu late (a(i. j)) at each iteration, and make hard decision d(/)(t.j). Then compare air)(i.j) to (i.j). If the number of disagreem ents is less than a certain num ber. stop the iteration because no significant improvement can be made. Also, it is possible to build A’i row SISOs and .V 2 column SISOs. All of them can operate in parallel because of the independence of soft inform ation used by each SISO. Clearly, the inner detector (column SISOs) and the outer detector (row SISOs) exchange the soft information on the auxiliary symbol Vj(i) via iteration. Although both the inner FSM and the outer FSM are ID systems, the iteration naturally helps to ex ploit the embedded 2D structure that is not covered by either the inner FSM or the outer FSM individually, and finally to yield a better solution than a non-iterative concatenated detector. By ignoring the internal structure of the iterative concatenated detector, it has the same interface as a regular SISO for a ID system: the soft input and soft output about a(i.j) and x(i,j), except for the 2D property of these quantities. Therefore we refer to the iterative concatenated detector as a sim ple 2D SISO algorithm (Col- Row version) illustrated in Figure 3.3(b). This 2D SISO algorithm is applicable to 34 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. any 2D d ata detection problem. Unlike the ID SISO algorithm for an isolated FSM. this 2D SISO algorithm is sub-optimal even for an isolated 2D system because the way it exploits the 2D ISI structure is not optim al. Applications in the following sections will show th a t this 2D SISO algorithm actually performs close to the MLPE performance. Similarly, the row-col version of the simple 2D SISO algorithm can be readily developed. Both the col-row and row-col versions can yield the soft output about a(i.j). A lthough th e underlying concatenated models are equivalent, it is not nec essary for these two simple 2D SISOs to yield the same soft output about a(i.j). Actually it is expected that they do not generate the same soft output due to the suboptim ality of either version. Moreover there is no way to tell which version is better than the other. Therefore it should be beneficial to combine their soft out puts. Consequently a composite 2D SISO algorithm illustrated in Fig.3.4 consists of these two versions of the simple 2D SISO algorithm . It is easy to see that these SIi(x(iJ)) SIi{a(Lj)) col-row row-col comb row SISO col SISO COL SISO ROW SISO Figure 3.4: A composite 2D SISO algorithm. two submodules can operate simultaneously and independently. After the operation is accomplished in both submodules, a soft information combiner produces the final soft output via A PP version: P0(a(L j)) = P l Q (a(i. j))P 2 Q (a(i, j)) (3.7) MSM version: Ma(a (i,j)) = U l Q (a(iJ)) + M*(a(z\ j)) (3.8) 35 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. and the hard decisions are made by thresholding the final soft output. A more sophisticated combining rule is discussed in Section 5.6. which is expected to provide a better performance. 3.2.3 A p p lications 3.2.3.1 The M inim um D istances for 2D IS I/A W G N Channels Although the M LPE is the best achievable performance for the d ata detection prob lem in a 2D ISI/AW GN system, its im plem entation is prohibitively expensive. Just like the MLSE performance [Fo72. Fo75. Ye87] to a ID ISI/AW GN channel, it is helpful to find an approximation of the MLPE [Ch96, ChChThAn99] and use it to evaluate algorithm s. Similar as in the ID case [Fo72], the crucial param eter to cal culate the M LPE performance bounds is the m inimum distance dmin [Ch96]. Define an error page between two data page At and A 2 as E =f {e(L m )}q.m) = A! — A 2, in which the error symbol e(l. m) € A A = {-4* — Aj, 0 < i.j < M — 1|.4,-. Aj € A}. For example, when A = {0 . 1}, A A = { —1, 0, 1}. It is easy to see th at |AA| < 2 In association with this pair of data pages, define the distance between them as Minimize this distance over all possible non-zero error pages, we obtain the dmm: This definition is exactly analogous to that of the ID case [AnFo75]. Roughly speak ing. the smaller dmin is, the more severe the 2D ISI is. For a more rigorous treatm ent of the performance, please refer to [Ch96j. Unfortunately, the approaches applied to the ID case to minimize the sufficient error sequence set [AnFo75, Ye87. \aFo91] Actually one can see th at finding dm in in (3.10) is as hard as finding the MLPE in (3.4). Alternatively, it is easier to find a constrained dmin defined as: (3.9) (3.10) cannot be applied to the 2D case because of its lack of a naturally ordered index. m tn min mm size of E 0 to satisfy the condition E # 0. Note that since cf(E) = d{ —E). it is sufficient to set e(0 .0) > 0 to find d'min. W ith these assumptions, the constrained dm jn problem is transformed into a d ata detection problem in (3.4) which can be efficiently and effectively solved by the metric-based composite 2D SISO algorithm above mentioned. By only considering 2D ISIs defined in (3.12) with binary input alphabet {0. 1}. the normalized {i.e., ||/ ( L j )||2 = 1) d 'm- m is calculated for different values of a and b by setting Im large enough and the page size 30 x 30. The results are quickly obtained and plotted in Fig.3.5 along with corresponding ■ ’In [ChCh98], the iterative detection yields a performance that is just < 1 dB worse than the computed bounds given in [Ch96|. Consider the sub-optimality of the iterative detection, it is fair to say that d' m• is a good estimate of dmjn. b 1 b (3.12) if (i.j) = (0. 0) if ( U ) # ( 0, 0) (3.13) 37 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. error patterns which achieve d'm in in various (a. 6) regions. By running the exhaustive (+) (+) \ / (+) 0.9 - o,8 J ( + dmin 0.7. ‘ 0.6 0.8 m in (dmin) = 0.35 Figure 3.5: Normalized f/'m in and corresponding error patterns for the 3 x 3 2D ISI. search algorithm on a much smaller page size, no conflict has been found. It clearly shows that the iterative approach is much more efficient than search algorithm s and also appears to be very reliable. From the results, one can see th at ail the rfm in error patterns displayed are centrally anti-sym m etric. This situation is analogous to that in ID ISI channels [AnFo75. YaFo91]. Therefore, we conjecture that the dm jn error page pattern for a 2D ISI with a binary input is centrally symm etric or anti symmetric. We also observe that d'm in reaches a minimum value of 0.35 when a = 1/2 and b = l/\/2 . It is interesting to note that this "worst" 2D ISI channel is the outer product of the worst 3-tap ID ISI channel: ( l / \ / 2 , 1. 1 /\/2 ) [Pr95. pg.601]. thus it is separable. We believe th at this observation should not be treated as a coincidence, but a superficial appearance of a profound connection between the ID system and 2D system although this connection is unclear yet. 38 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 3.2.3.2 D a ta D etection for Page-O riented O ptical M em ory Signal and N oise M odel Currently, there is a great interest in parallel recording and retrieval techniques for the next generation high capacity storage systems. The page-oriented optical memory (POM) system is a promising candidate because of its potential for simultaneously achieving high capacity, fast d ata transfer and 2D parallel access. However, real-world POM systems operating near their capacity are subject to numerous sources of noise and interference arising from the optical system itself. Specifically, the blur resulting from the low-pass nature [Go68] of the optical system causes neighboring bits in a d ata page to overlap during the retrieval procedure. This pre-detection retrieval procedure can be modeled by a 2D ISI channel: s(x,y) = ^ a ( i . j ) h ( x - iA .y - j A ) (3-14) dj) where A is the vertical and horizontal detector (e.g.. CCD pixel) spacing, and the effect of interference is captured by h(x,y), which is assum ed to be zero outside the region {(x. y) : |x| < (L + 1/2)A , |y| < (L + 1/2)A }. The optical-to-electrical conversion process is assumed to take place by means of an array of detectors (e.g., CCD) th a t integrates the intensity of the signal over both tim e and space. Further more. it is assumed that a detector is centered on each pixel location and the fill factor is unity (i.e.. no spatial guard bands between detectors). For a POM system using incoherent signaling, the function s(x,y) in (3.14) characterizes the intensity function directly. Let x(i.j) represent the output of the (i.j)-th. detector, we obtain a discrete-space model through Lc Lr x ( i ,j ) = ~ l- j ~ m ) (3.15) l— — Lc m = — Lr / A /2 /-A /2 I h(x + IA, y + rnA)dxdy (3.16) •A /2 j - A /2 This results in the same 2D ISI model as in (3.2). The blurred output from the 2D ISI channel is also corrupted by observation and post-detection noise, which can be well m odeled as an AWGN. In this dissertation, we will only consider the 2D ISI/AW GN model for a POM system . O ther signal and noise models (e.g.. coherent 39 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. field noise, shot noise, etc.) are possible [ChChNe99]. Since the IDD is based on soft information , it is straightforward to adapt it to other signal and noise models. In this section, specific examples of the 2D-ISI/AW GX channel are considered with a binary signaling of intensity of 0 and 1, and a square 2D ISI pattern with Lr = Lc = 1. This assum ption results in a concatenated model consisting of a 64- state inner FSM, a 4-state outer FSM and a block interleaver of the page size. The performance measure - bit-error rate (BER) - is plotted against the SXR defined by XZ(t.j) / 20 -3)1-°2’ Three 3 x 3 symmetric 2D ISI channels (Table 3.2) as in (3.12) are used to test the performance of 2D SISO modules. The first two channels are Channel a b A 0.0327 0.181 B 0.0993 0.352 C 0.5 0.717 Table 3.2: Param eters of 2D ISI channels used in simulations. truncated Gaussian blur channels [XeChKi96] which arise from the real-world POM systems. The first channel (Chan-A) has a Gaussian blur param eter a< , = 0.45. and the second (Chan-B) has a& = 0.623. The last test channel (Chan-C) is the theoretically "worst7 ' channel found in the last section. D e te c tio n A lg o rith m s As the best achievable performance, the MLPE bounds developed in [Ch96] are treated as the baseline for the performance evaluation. For comparison, we also present the results for the following conventional algorithms: • Thresholding detector: Let m: =f E{c(i, j)} , the threshold decision (TH) rule is: d(i,j) = 1 if z(i,j) > m ~: otherwise, a(i,j ) = 0. Given the channel in (3.12)3 and the binary input alphabet {0. 1}, the 2D ISI gives finite values of x(i.j). In the case of AWGX, mz = (1 + 4a -t- 46)/2. W hen a(i,j) = 0 the largest possible value of z(i, j) is 4a + 4b. When a{i,j ) = 1. the smallest possible output is 1. W hen I > 4(a + & ), the performance of TH does not have a BER floor. However, when 1 < 4(a -I- b), there exists an error floor. Given 3 Without loss of generality, we assume a > 0 and 6 > 0. 40 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. the value of i, let J, denote the minimum value of j such th at ia + jb > m:. the BER floor is Therefore when the 2D ISI is severe, it can be expected th at the threshold detector will fail. • M M S E e q u a liz e r: Denote v0 = v — S{c}. The m inimum mean square error (MMSE) equalizer is defined by a (2Q + 1) x (2Q + 1) linear filter g(irj) and the operation The filter coefficients are obtained by solving the W iener-Hopf equations Kzg = kza. with K z = E{z0(i.j)zt 0(i.j)} and kza = E{z0(i. j)aQ(i, j)}. The lin ear estim ate of a0(i,j) given by a0(i.j) is then threshold detected against a zero threshold to determine the final bit estim ate. Specifically, a(i.j) = 1 if > 0. otherwise a(i.j) = 0. For more discussions about the MMSE equalization for 2D ISI/AWGX channels, please refer to [ChChXe99j. • D ecision feedback equalizer: The MMSE equalizer discussed above is uti lized in conjunction with a simple threshold decision rule. It is possible to replace this rule with a simple iterative decision-making process [XeChKiOGj. The first step of this process is to apply the threshold to the MMSE output page and produce an estim ate a(i.j). Given a(i,j) = 0 and other pixels with values of a(l.m). we calculate the 2D ISI output at pixel (i.j) and denote as x0(i.j). Similarly, xi(i.j) is obtained conditioned on a(i,j) = 1. Then a up dated decision is made based on the distance between z(i.j) and x0(i,j) and xi(i.j). If \z(i.j) - xo(i.j)| < |^ (h j) - then a(i.j) = 0; otherwise = 1- This iteration can performed certain tim es or until no further change takes place. We refer to this algorithm as decision feedback equalizer (3.17) Q Q l= - Q m = —Q (DFE). 41 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. • D ecision feedback VA: Although the standard V'A [Fo73] is not applicable to the 2D ISI case for the lack of a naturally ordered index, the decision-feedback Yiterbi algorithm (DFVA) [HeGuHe96] adapts the VA by imposing an ordering on the 2D index and achieves some reasonable success. As illustrated in Fig.3.6 for a 3 x 3 2D ISI channel, the DFVA runs a VA with a state consisting of a 2 x 2 symbol m atrix (e.g. state j includes symbols from row i and i 4- 1). The transition metrics are computed with the help of the hard decisions from fixed hard decisions from previous row processing row - 1 XX XXX row $ Q 0 # 9 row i + l * # * # * state j state j + 2 state j — I state j + 1 Figure 3.6: Decision feedback VA for the 2D ISI channel. the previous row (i.e.. i — 1) processing. A fter reaching the end of each row. the VA traces back the best path and makes decisions only on the row i. Recursively, the DFVA can utilize the hard decisions of row i to continue the above processing for the following row. Like other decision feedback algorithm s [EyQu88, DuHe89], the DFVA will suffer serious error propagation problem when the SNR is low. In addition, the decision feedback operation prevents the DFVA from being implemented as a parallel detector. Sim ulation R esults C h a n -A The simulation results for Chan-A are plotted in Fig.3.7. The nor malized d'min for this channel is 1. The performance of a simple 2D SISO algorithm with only 2 iterations virtually achieves the M LPE performance. It is easy to see that the A PP and MSM versions of the simple 2D SISO algorithm perform very closely to each other. The performance of threshold detector is poor, but no BER floor exists since 4(a + 6) < 1. However, the BER performance of the DFE (using 42 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. a 5 x 5 filter and 2 decision feedback iterations) and the DFVA is quite good as well. Although there is a ldB gain4 in SNR by the simple 2D SISO. it is difficult to justify given the complexity. Moreover, since the MLPE bounds represent the best achievable performance of any detector, the results show no room for further improvement by using the composite 2D SISO algorithm. \ 10' •> 10 Threshold DFE DFVA 10' — 2 D - 5 — APP-0-2 MSM-0-2 — • MLPD Bounds 10 O 1 0 2 0 1 0 15 o SNR(dB) Figure 3.7: BER performance of various detectors for Chan-A. Note that the suffix -/ is the number of iterations used. APP-0 or MSM-0 stands for the simple 2D SISO, and 2D4 for the algorithm discussed in section 3.3. C h a n -B Sim ulation results for the more severe Chan-B are plotted in Fig.3.8. The normalized d'min for this channel is 0.76. Because 4a + 4b > 1. the threshold detector is expected to hit a BER floor around 0.1. Both the DFE (using a 5 x 5 filter and 2 decision feedback iterations) and the DFVA algorithms perform 6dB worse than the M LPE bounds. However, 2D SISO algorithms perform very close to the MLPE bounds. Specifically, a simple 2D SISO algorithm performs < ldB worse than the MLPE upper bound by using only 3 iterations. Moreover, the composite 4 Note: from now on. the performance of various algorithms are compared at a BER of 10- '1 43 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 2D SISO algorithm using 4 iterations achieves the upper bound. This is the best achieved performance to our knowledge. Again, the MSM version works almost as well as the A PP version. 10 10 ' Threshold DFE DFVA 3 1 0 4 APP-0-3 MSM-0-3 1 0 t 1 — MLPD Bounds 1 _ - — APP-00— 1 1 o'3 _ -------- 5 10 15 2 0 25 SN'R (dB) Figure 3.S: BER performance of various detectors for Chan-B. Xote th at APP-00 or MSM-00 stands for the composite 2D SISO module. C h an -C The advantage of 2D SISO modules is expected to be more significant for Chan-C of which the normalized d'm in = 0.35. The results are plotted in Fig.3.9. This time, besides the threshold detector, the MMSE equalizer (using a 7 x 7 filter) fails at high SXR as well. The DFVA still functions properly despite suffering a lOdB performance degradation compared to the MLPE bounds. However, both the simple and composite 2D SISO algorithms perform close to the bounds by only 4 iterations. Specifically, a simple 2D SISO algorithm performs about ldB worse than the upper MLPE bound, while a composite 2D SISO algorithm improves this performance by 0.2dB. Again, the BER performance difference between the A PP and MSM versions is still negligible. In summary, the 2D SISO algorithms always provide the best results known for all three test channels. The MSM version always works as well as the A PP version. 44 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 10 1 10 o 10' '■ .x 3 10' -s— Threshold e — MMSE -a — DF-VA ■ * — APP-0-4 * MSM-0--1 . APP-OO-l - MSM-00-4 MLPD 4 10' 10' 2 0 2 5 3 0 10 15 SNR (dB) Figure 3.9: BER performance of various detectors for Chan-C. This is consistent with the conclusion drawn in [HaCoRi82j for ID cases. We also notice th at there is still a > ldB distance between the bound and the best achieved performance for Chan-C. It is possible to obtain this additional improvement by fur ther refining the iterative procedure an d /o r the concatenated system model. Some experim ents have been done to improve the perform ance by another 0.5dB by prop erly exchanging soft information between the two submodules inside the composite 2D SISO. Additionally, the 2D SISO algorithm s need to be simplified when applied to larger 2D ISI channels. In [ChCh99b] (see also Section 4.2). a reduced-state (RS) SISO algorithm is able to reduce the complexity significantly. In real-world systems, the entries on the edge of a 2D ISI pattern are usually relatively small. Thus the RS-SISO algorithm s can be used to adapt the 2D SISO algorithm s to channels with larger 2D ISI patterns. 3.2.3.3 Im age H alftoning The hum an visual system can be modeled as a low-pass filter which blurs the image observed [GrSl]. Also, laser printers generate distortions such as ”dot overlap" 45 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [PaXe95]. Consequently, when a digital image is printed out and observed by the human eye. we do not see the original image but a blurred image. Unlike the destructive blurring effect in the storage system , the blurring effect discussed above indeed enables the image halftoning. The basic idea in halftoning is to represent a gray-level pixel z(i,j) (on a scale of 0 = white and 1 = black) by a black or white pixel a(i,j) (i.e.. 1 or 0). If the distance between adjacent pixels is sufficiently small, the blurring effect discussed above averages black spots and white space, and perceptually approximates the original gray level z(i,j) with the blurred output x(i-j) = f(i-j) * a(i-j)- "'here {f(i.j)} is the 2D ISI model of the blurring effect. Given the original gray-level image Z and a halftoning image A. a cost function is defined as C (A ) =f ]C(,\j) ~ /(*'• J) * a(i-j)\2- s a special case of the DLM problem in (3.4) [ChChOrCh98], the least squared (LS) image halftoning problem minimizes this cost function. Therefore, the 2D SISO algorithms are applicable to the LS halftoning problem. A 512 x 512, 256 gray-level version of the Lena image0 (Fig.3.10) is selected as the test image. A simple threshold halftoning rule is a(i,j) = 1 if z(i,j) > 0.5: otherwise, a(i.j) = 0. The resulting halftoned image shown in Fig.3.11 is perceptually unacceptable. The DFVA [HeGuHe96] can also be applied to the LS halftoning problem. Although the halftoned image using the DFVA (Fig.3.12) is much better than the one using the simple threshold rule, the visible horizontal line pattern in resulting image clearly shows the row-by-row processing m anner of the DFVA. To apply the 2D SISO algorithm , the blurring effect is modeled by a 3 x 3 2D ISI with coefficients / 0.2219 0.1439 0.0355 N 0.1439 0.0980 0.0306 (3.19) y 0.0355 0.0306 0.0174 This filter is inspired from the filter used in [Wo97] without any effort to optimize. W ith uniform metric initialization of the pixels, the halftoned image after 3 iter ations is shown in Fig.3.13. The cost function associated with this approach is °This printed image is halftoned by the printer built-in halftoning algorithm. A tilted grid pattern is visible. 46 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Figure 3.10: Printer halftoned Lena image with 256 gray-levels. approximately 40% less than th at of the DFVA. This image has much less annoy ing "regular" patterns than the DFVA one, therefore is more perceptually pleasing. However, some patterns still exist in this image, especially on the face and at the right portion. One perceptually undesirable property of the halftoned image is the distinctive pattern in areas of constant gray-level. The soft-input nature of the SISO processor allows one to control these patterns by biasing the algorithm toward a par ticular reference image. For example, an initial bias toward a gray pattern suitable for a particular imaging device may be appropriate. Random biasing can be used to alleviate the patterns in Fig.3.13. On the first iteration, we set the a-priori metrics of 'Vj(i) = Ylf^-Lr ^Ii[a(bJ — 0] "'here M;[a(z, j ) = 0] is a random number uni formly distributed in [0, r] and 5Z.4=o,i Mi[a ( b i) = -4] = r - Note th at r is a design parameter. This random initialization obviously has the effect of biasing toward a random halftone to alleviate the distinctive patterns. This provides a clouding of the halftone which is partially removed with each iteration. Fig.3.14 shows a halftone version of Lena using 3 iterations and the random biasing with r = 30000. Compared to Fig.3.13, much less distinctive patterns can be seen. Therefore this im- 47 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Figure 3.11: Threshold Halftoned Lena image. Figure 3.12: Halftoned Lena image using the DFVA Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Figure 3.13: Halftoned Lena image using the 2D SISO with the uniform initialization. Figure 3.14: Halftoned Lena image using the 2D SISO with a random initialization Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. age is more perceptually preferred. However, com pared to the halftoned image using error diffusion [PaNe95] (Fig.3.15), the perceptual quality of the image in Fig.3.14 Figure 3.15: Halftoned Lena image using error diffusion. is still inferior. Obviously, using a larger filter {e.g., 7x7) would enable improving the performance of the 2D SISO halftoner significantly. However in this case the 2D SISO algorithm has to use the RS-SISO submodule [ChCh98b]. In [ThChChOr99], a more sophisticated initialization technique and iteration structure are used to get a high quality halftoned Lena image in Fig.3.16 which is even better perceptually than the error-diffusion halftoned Lena. 3.3 A n Iterative D istrib u ted D etecto r 3.3.1 A B ayesian N etw ork C hannel M od el In the last section, a concatenated channel model is proposed to "index" the 2D ISI in a ID fashion. W ithout trying to index the 2D system, we can view each observation pixel z{i.j) as the output of an ISI processor. This ISI processor has a major input a(i,j) and some auxiliary inputs (interference). Thus the 2D ISI/AW GN channel is 50 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Figure 3.16: Halftoned Lena image using a sophisticated iterative scheme. naturally decomposed into an Ni x JV 2 ISI processor array, modeled by a Bayesian network in Fig.3.17. This Bayesian network can be organized into three layers. The layer 1 is the input layer consisting of the data array A . The layer 2 is the ISI output layer and the layer 3 is the observation layer. Illustrated by the connections between the layer 1 and the layer 2, the ISI processor maps subsets of the input array into ISI outputs. Mathematically, the 2D ISI output in (3.2) can be reformulated as: as defined in Table 3.1. The neighborhood N(i,j) can take on values from the space 2 into the observation in the layer 3. Obviously, this multi-layer Bayesian network model for the 2D ISI/AW GN channel is far away from cycle-free. Consequently, no optimal BPA can be derived for this network. However, it is possible to obtain an ad main input auxiliary input — — a(l,m) (3.20) where the neighborhood N(i,j) = S(i,j) — a(i,j ) and the support S(i,j) n =f ^lt(2i’ --ri)(2L 0+i) i] Then an observation process maps the ISI output in the layer 51 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ISI processor x(i.j) laver2 laver 1 Figure 3.17: A Bayesian network model for the 2D ISI/AYVGX channel. hoc iterative BPA. For the simplicity of presentation, only the case of Lc = Lr = 1 is considered in the following discussion. 3.3.2 A n Itera tiv e D istributed D etecto r By ignoring the cycles, an iterative BPA for this Bayesian m odel can be derived and illustrated in Fig.3.18. In the full}' connected BPA. the soft inform ation about the input a(i.j) is stored and refined at the (z.j)-th soft inform ation processor in two steps. First, the soft information about a(i.j) is combined together based on the local evidence z(i.j) to generate the soft inform ation about the support S(i.j) via: P„(S(i,j))Spr(S(i.j).;(i.j)) = P r(s (i.j)|5 ( i.y » b [ P i(a(/,m )) (3.21) a(l.m)£S(i,j) Denote S (i.j) =f {■S'(/, m ) : |/ — i| < Lc. \m—j\ < Lr}. Then this soft information is marginalized to generate updated soft information about input symbols a(i.j) via: Pi(a(i.j)) = Pr(a(i.j)\S(iJ)) [ J P 0(S (/.m )) (3.22) S(i.j) S(l,m)eS(i,J) 52 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. soft inform ation • C • > . ^ ^ 3 ^ > 3 ^ X 3 w ^ o b se rv a tio n ^ • soft inform ation storage and processing Figure 3.18: An iterative distributed detector for 2D ISI/A W G X channels. However, this fully connected BPA is fairly complicated due to the massive con nections am ong the nodes. When the value of a tap in the 2D ISI is relatively small, the m arginalization operation associated with this tap is relatively unreliable. Fortunately, for the real-world 2D ISI channels (e.g.. bell shape), the taps located at the central part usually have the m ajority of the power. In such cases, we can significantly simplify the m arginalization operation to: Pi°(o(i,3)) = Po (5 (i,J» (3.23) S{i,j):a(i.j) Putting the combining step and the simplified m arginalization step together, the soft inform ation processing step can be w ritten down as: P ? M i . j ) ) = J 2 Pt(z(i.j)\.X(i.j).a(i,j)) n p !0 < “ ('O » (3.24) A ' ( « . j ) e n a ( / , m ) e j V ( i , ; ) 53 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. which is the updating operation in the z'-th iteration of a distributed SISO algorithm . This iteration is initialized with P[0 ) = P r(a (z .j)). the a-priori information about a(i.j). Many iterative algorithms exhibit a trade-off between the convergence rate and the performance. Due to the approxim ation used in (3.24). a slower convergence would be helpful. This motivates the following filtering process at each pixel: P ! " l,(a (i.i)) = (1 - 3 )P !"(a(/, j)) + 3 P (3.25) The filtering param eter 3 €E (0.1] determines the bandw idth of a single-pole low-pass filter which rejects abrupt changes in the updated P ^ a l z .j ) ) . We refer to this spe cific iterative' SISO algorithm as 2D distributed d ata detection algorithm (2D- 4 ) in [ChCh.\e98] since a processor can be built at each pixel (i.j) and run independently. The structure of these processors is illustrated in Fig.3.19. In [ChChXe98]. a more T ... soft inform ation P (a(z.j)) ► processor ► x(l — 3) Figure 3.19: Block diagram of the 2D4 algorithm. intuitive derivation of the 2D4 is presented. The derivation presented here is pre ferred since it fits into the more general framework based on the Bayesian network and the BPA. In this situation, the effect of iteration can be readily observed. After the zeroth iteration, we get Pj°*(a(z. j)) = Pr(z(i. j)\a(i, j)). a soft information about ci(i.j) only based on a local evidence z(i,j). After the first iteration, the information from a larger evidence set {z(l, m) : |/ — zj < Lc, |m — jj < Lr} is collected into P{ 0l](a(i.j)). although in a suboptimal way. More iterations used, the information from a larger range of observations is considered. The iteration may be continued until some stopping criterion (e.g., [P£*^(a(z, j)) — P ^ (a (z ,j))| < e) is met or for a fixed number of iterations. In either case, if the stopping occurs after Im iterations, the decisions are made according to: a(i.j) = 0 if P^/m*(a(z.j) = 0) Pr(a(z. j) = 0) > Po",)(a(i.j) = l)Pv(a(i,j) = 1), otherwise a(i,j) = 1. Although the 2D4 is 54 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. presented in the probability-based manner, it is easy to write down its metric-based version using the translation rule developed in Section 2.1.2.4. A M o re S o p h istic a te d C o m b in in g S ch em e Since the ideal goal of the 2D4 is to converge to the likelihood of a(i.j) based on the entire observation page Z. it is desirable to set the combining operation in (3.21) based on a larger region of the observation data to achieve a stronger combining effect. It is equivalent to assemble more than x{i.j) into each node in the layer 2 in Fig.3.17. The simplest case is to use z(i.j) in (3.21). W ith only slightly higher complexity, the combining operation can be executed based on z+(i,j) = {z{i,j),z(i ± 1 ,j).z(i,j ± 1)}, i.e.. the set of observations located on a cross 7Z(i,j) whose definition is shown in Fig.3.20. In this n ( i j ) V(i-J) Figure 3.20: Several pixel sets defined for the 2D4 algorithm . figure, we also define sets 'H(i.j) and V(i.j). Then we can define the set £{i.j: m) = H(i + m. j) U 7i{i — m, j) U V(z, j -I- m) U V(i.j — m) (3.26) Thus, a more sophisticated combining scheme can be obtained as: Po(S(i.j)) = Pr{S(i,j). z+{i,j)) = Pr(z+(f, j)|5y) Pr{S{i. j)) = c Y . Pr(z+(i.»|£(i,j:2),S(t,j))Pr(S(i,j)) ‘ = C E I I PrWi,m)|S(/,m))Pr(5(i,j)) £(ij;2) ( / . m ) € 7 2 ( i j ) 55 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ( =’ c lP r ( j ( i,j ) |S ( i.i) ) J ] P ,(a(l.m)) \ a(l.m)€S{i.j ) X F I Y , P H :(,i + l-j)\S(i + l.j)) l= ± 1 71(1+21.j) x E l H Pr(z(i.j + m)\S(i.j + m)) (3.27) m —± 1 V(i.j+2m) The constant c is independent of S{i.j). The equality (i) follows from the conditional independence of the observations in z+{i.j) and the fact th a t z(i -hl.j + m) is only dependent on S(i + I.j + m) C S{i.j: 2) U S(i,j). T he equality (ii) follows from the independence am ong H{i ± 2 .j) and V{i,j ± 2). Clearly, the first term in the parenthesis after the equality (ii) is exactly the simple com bining operation in (3.21). While this combining operation does not require significantly more com putation than (3.21). the attem pt to com pute the soft information about S(i.j) based on larger regions of the observation results in an exponential increase in complexity. R educed-C om plexity C onnection Shown in Fig.3.18, the massive connections between a processor and its neighborhood processors determ ine the complexity of the 2D4 algorithm . The larger the size of X{i.j) is. the more complicated the 2D4 is. Fortunately, for most low-pass channels, the ISI coefficients rapidly decrease in magnitude at the edge of the neighborhood. For exam ple, even in the relatively severe Chan-B in Table 3.2. / ( I . l) //( 0 . 0) = 0.0993 is relatively small. Thus, we consider om itting some entries of N(i.j) to simplify the com putation. For each en try that is om itted, the connection complexity is reduced by l/j-4 |. In Fig.3.21(a). the '‘full connection" (FC) structure suggested by the original derivation is a special case. By om itting the 4 corner-to-center connections and using the input mean ma in the calculation of P r(^(t. j)\S{i, j)), we obtain the ‘ *no-corner connection without feedback" (XCC) structure shown in Fig.3.21(b). Replacing the input mean in XCC by the tem porary hard-decisions which are made after each iteration, the "no-corner connection with feedback" (XCCF) structure in Fig.3.21(c) is just an application of the decision feedback technique {e.g.. [DuHe89]). In both XCC and XCCF struc tures. even for the binary input, the complexity is reduced 16 times compared to the FC structure. Obviously, it is expected that this com plexity reduction is realized at 56 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. soft inform ation input mean hard decision * * k- • • • • • • • • • • (a) (b) (c) Figure 3.21: Several connection structures defined for the 2D4 algorithm. the cost of performance an d /o r convergence rate because the combining strength is weakened. Likelihood R atio In the binary case, only the ratio of P0(a (/./) = 1) and P0( a (i.j) = 0) is necessary to make a decision. T his fact can be exploited to reduce the com putational and storage requirements of the 2D4 algorithm two times. Specifically, we can define R,(il(o(i. J)) = P f°(a (/.j) = 1)/P !'> (a(i.j) = 0) (3.28) RV’M ' J ) ) " P o t(a (,i2) = D /P ? )(o (i.j) = 0) (3.29) The filtering operation can be rew ritten as: R!'+ ,,(a (i.;)) = (1 - J ) R f ( a ( ;,j) ) + S R g> (a(U )) (3.30) and the soft information processing can be simplified as: Pi'’(a(/.j)) = £ Pr(.-(i.j)|.V(i,»,a(i.j)) [ J R,«>(«(f,i)) (3.31) A ( i . j ) € f 2 a (l,m )£ \(i,j) a(l.m) = 1 On average, the number of multiplication operations required is reduced by a factor of 2. Also, the decision rule is modified to be a(i.j) = 1 if R *,/v)(g(z, j)) > 1; O f Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. otherwise a(i,j) = 0. By only storing R |^(a(f. j)), the memory' requirements for the instantaneous soft inform ation quantities are cut in half. 3.3.3 D a ta D etectio n for th e 2D IS I/A W G N C hannel 3.3.3.1 C onvergence Property The convergence rate of the iteration in 2D4 determ ines its time complexity. Xo analytical way is known to predict this convergence rate. However, certain results can be obtained through numerical experiments. Choosing Chan-B in Table 3.2 as the test channel, the BER curves for various values of 3 are plotted in Fig.3.22 given IXY =f 1 / o~ = 27dB. Obviously, the convergence rate is dependent on the value of 0 .3 1 0 ' 1.2 10' 20 30 -,i 1 0 ' Iteratio n N u m b er Figure 3.22: Convergence property of the 2D4 algorithm . The inset is a magnified version of the lower portion. 3. More specifically, when 3 < 0.5, the convergence is slower with decreasing 3, although the performance after convergence is similar. For example, for 3 = 0.1. convergence is achieved for i > 50. but for 3 = 0.5, convergence occurs for i > 15. This behavior is reversed for 3 > 0.5 - i.e., the performance instead of the convergence rate is more sensitive to varying 3 • For 3 > 0.5, convergence occurs at approximately 15 iterations with the steady-state BER varying by up to 25% (see 58 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. the inset in Fig.3.22). Also, oscillations in the BER results are observed after a certain num ber of iterations, especially when 3 > 0.9. Based on these results, we use 3 = 0.3 in the following simulations as a reasonable choice with no more than 20 iterations required for Chan-B. Of course, the convergence rate is also dependent on the channel itself. Numerical experiments show that the severer the channel is. the slower the convergence is. For example, using the same value of 3. the less severe Chan-A in Table 3.2 needs only 5 iterations before converging. 3.3.3.2 Im pact o f Reduced C onnection C om plexity The im pact of reduced-complexity connection structures is investigated by compar ing the performance of the 2D'1 using different structures. Fig. 3.23(a) shows that the impact of the connection complexity on the convergence rate is negligible. However, a certain performance degradation due to complexity reduction is observed for both channels. Specifically, at a BER of 10-4 , the XCC scheme suffers a loss of 0.4 dB and 2.0 dB in IXV on Chan-A and Chan-B. respectively. However, with the help of hard decision feedback, there is virtually no performance degradation for Chan-A. while the degradation is reduced to only 0.3 dB for Chan-B. Actually, the XCC 2D4 algorithm still outperforms the DFYA presented in Section 3.2.3.2. 3.3.3.3 2D ISI M itigation The purpose of developing the 2D4 is to efficiently solve the d ata detection problem associated with the 2D ISI channel. The pixel-wise processing facilitates a highly parallel com putational architecture for the implementation of the 2D4. Compared to the 2D SISO algorithm developed in the previous section, the 2D4 is more suited to the nature of 2D systems due to its 2D distributed structure, therefore it is potentially much faster than the 2D SISO algorithm . The MSM version of the 2D4 is applied to the 2D ISI channels listed in Table 3.2 and the simulation results are plotted in Fig.3.7 and Fig.3.8. For Chan-A with 5 iterations, the 2D4 (3 = 0.3) works as well as the simple 2D SISO algorithm and virtually hits the MLPE bounds. For more severe Chan-B, the 2D4 (3 = 0.3. I m = 20) performs 2dB worse than the simple 2D SISO algorithm while still much better than the DFYA. and DFE. However, when applied to Chan-C, the 2D4 does not work th at well (no result is presented). This 59 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ■ fully connected - 5 ---- no-comer couuectetl w/o fo«dI>*u:k : O no-corner connected w/ feedback ; 1 0 - a -a 1 0 0 10 20 30 30 Ite ra tio n N u m b er (a) fully connected - no-corncr connected w/o feedback a no-comcr connected w/ feedback .. C hannel A C h a n n el B . 20 24 IN V (d B ) 28 30 (b) Figure 3.23: (a) The convergence properties for the Chan-B at IXY = 27 dB. (b) BER performance for the Chan-A and Chan-B. The num ber of iterations used is 5 and 20 for Chan-A and Chan-B. respectively. is because Chan-C is so severe th at the approxim ation used in the derivation of the 2D 1 fails it. Fortunately, most real-world PO M systems are similar to Chan-B or even better. Hence the advantages of the 2D4 are substantial. 60 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. C hapter 4 Iterative D etection for C om plexity R ed u ction 4.1 C om plexity P roblem In the previous chapter, the IDD technique is applied to the 2D d a ta detection. For such problems the efficient and optim al (i.e.. MLPE) solution is unknown. For ex ample. the LUT gives the MLPE but has a complexity of |«4|A‘ |.4.|z 'rZ 'c. We call such a problem m athem atically hard (MH) since the complexity of the known optimal solution is exponential in the problem size .V. As an alternative of the too-expensive optimal approach, the IDD is used to efficiently solve the MH problem in a near- optimal manner. Unlike the MH problem, the com putationally hard (CH) problem has a known efficient optimal solution. It is well known that the YA for a ID FSM is optim al (i.e.. MLSE) and has a complexity of K \A \L which is only proportional to the problem size K . The fundam ental difference between the complexity of an MH problem and a CH problem can be described rigorously by the XP-completeness theory [GaJo79]. Moreover. Fig.4.1 can be helpful to understand the terminology used in this chapter. Although the CH problem is m athem atically easier than the MH problem, it still can be difficult. For example, when the length of ISI memory L is long or the size of input alphabet \A\ is large, the complexity index of the YA - the number of states \A\L will ju st grow beyond the feasible com putational capacity at an exponential rate. Unfortunately, many real-world systems do have similar CH problems embedded. Since it includes two recursions similar to the YA, the SISO algorithm for a simple FSM faces the exactly same com putational difficulty as the 61 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. hardness A iterative detection easy Figure 4.1: Hardness of problems. YA does. Again, it is possible to apply the IDD technique efficiently and near- optimally to solve a difficult CH problem. In this chapter, two specific examples will be discussed. As with the YA. the complexity of a SISO algorithm grows exponentially with the memory length L of the FSM. e.g.. the number of taps in an ISI or the constraint length of a convolutional code. For instance, when the MSM FI-SISO algorithm is applied to a 10-tap ISI channel with QPSK signalling, its complexity is determined by the num ber of states of the FSM. i.e., 49 = 262.144. This makes the detector based on the MSM FI-SISO algorithm practically infeasible. From the reliability information carried by the soft information {e.g., an A PP ratio R(a) =f P (a = 1)/P (a = 0)). it is possible to identify and detect certain information quantities on which the reliability measure has reached a given threshold {e.g., R(a) > 0.9 or R(a) < 0.1) early in the detection process. In [FiGe96. FrKs98], this early detection technique is used to reduce the complexity of SISO algorithm s. By employing the currently-available hard decisions [LeHi77. BePa79]. the decision feedback technique is widely used for complexity reduction in forward-only hard-decision algorithms. 4.2 R educed-State SISO A lgorithm (brief) 62 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. e.g.. the RSSE in [EyQu88] and the DDFSE in [DuHe89]. In [FiGe96], this m ethod is also applied to a forward-only SISO (i.e.. L2VS-SISO). In [ChCh99b], a so-called reduced-state (RS) SISO is obtained by carefully applying this technique to the bi directional SISO algorithms. Truncated states are properly defined and tentative hard decisions are generated. Also, the desired soft output is yielded in a simplified way. The resulting complexity reduction is exponential to the number of feedback taps. The RS-SISO has the same in p u t/o u tp u t interface as the standard SISO algorithm . In addition, the RS-SISO algorithm may feed its own soft output back to its input port (which we refer to as self-iteration) to refine the soft output. Therefore, not only it can be used effectively as a reduced-complexity SISO module in detection networks, the iterative RS-SISO algorithm can also replace a hard-decision algorithm (e.g.. YA. DDFSE) in the case of sequence detection and give robust and excellent performance. The RS-SISO is discussed in detail in [ChCh99b]. 4.3 Iterative D etection for Sparse ISI C hannels 4.3.1 Sparse ISI C hannel and E x istin g A pproaches Substantial effort (e.g.. [EyQu88. DuHe89. ChCh99b]) has been made to reduce the detection complexity for generic ISI channels at a cost of reasonable performance degradation. Nevertheless, a special type of ISI channel - sparse ISI (S-ISI) - de serves separate consideration since those complexity-reduction algorithms are usually not applicable to it. An S-ISI channel has an impulse response with only several short pulses spreading over a very long period. The S-ISI channel model is applica ble to various data communication systems. In a high frequency (HF) radio channel (3-30 MHz) [Go66. Wa.JuBeTO], the signal is transm itted from one site to another by ionospheric reflection. This phenomenon usually results in an S-ISI channel since more than one ionospheric reflection path exists in practice and the spread in arrival time could be on the order of 3-5 ms. A fixed point-to-point high-data-rate wireless connection can be well modeled as an S-ISI channel because of the m ultipath trans mission and the short symbol time. The S-ISI channel model can also be applied to certain mobile radio channels [Pr91]. In addition, certain convolutional encoder with sparse generator polynomials [BeBeMa98] can be interpreted to intentionally embed 63 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. an S-ISI in the transm itted data sequence. Two such S-ISI channels are shown in Fig.4.2. ionosphere \ X x X a high frequence channel a fixed point-to-point high-data-rate wireless connection Figure 4.2: Two examples of the sparse ISI channel. The baseband equivalent model of a general ISI channel with memory length L can be expressed as [Fo72b] where x k is the channel output at time k. is an additive white Gaussian noise (AWGX). It is straightforw ard to adapt the SISO algorithms developed in Section 4.3.2 for other types of independent noise. The output sequence of a data source {«i. a -2. ■ ■ ■. cik} is assumed to be independently drawn from the .\/-ary alphabet A — {.40. .4.1. • • •. } with a-priori probability Pr(a/t). However, for an S-ISI channel, while L is fairly large, most of the ISI channel taps in {/o- f\, • • • . /*.} are zero-valued. For the simplicity of expression, we define the i-th non-zero tap by a 3-tuple (i. p(i). f(i)) where p(i) represents the original index of this non-zero tap in the S-ISI channel and f(i) is its value, i.e., f(i) = fp(i). Also, we denote the number of non-zero taps in an S-ISI channel by Ls + 1 with Ls < g ; L. From now on. unless otherwise stated, the term "entry" will only indicate the non-zero tap. Since / 0 and L (4.1) i = 0 64 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. ft m ust be non-zero, we can express these two entries as (0 ,0 ,/ 0) and (Ls. L. fi) , respectively. The channel output can be re-expressed as: x k = ^ 2 f(i)cik-P(i) (4.2) t = 0 In order to describe an S-ISI channel mathematically, some definitions are re quired. First, we define a distance function d(i,j ) =‘ |c?i(/.j )| where di(i.j) =f p(i) — p{j). Then, entries q and z2 are called adjacent if d(ii,i2) = 1. The z-th en try is called isolated if it is not adjacent to any other entry, i.e.. d(i. i — 1) > I and d(i -4 - 1. i) > 1. A set of entries {z : q < i < z 2} is called grouped if d(i.i — I) = 1 for z 'i -I- 1 < z < z 2. An S-ISI channel is called regular1 if the en try set {i : nlg 4- 1 < i < (n + 1 )lg — 1} is grouped for n = 0.1. • • •, Ng — 1 (i.e.. Xglg = Ls + 1) and d(i. i — 1) = dg > 1 for z = lg, 2lg, • • •, (Ng — 1 )lg: otherwise, it is called irregular. In a regular S-ISI channel. lg is called the group size. dg is called the group distance, and .V 3, the number of groups. An S-ISI channel is called discrete if all its entries are isolated. An S-ISI channel is called simple if it is regular and discrete. For the future use. we also define the neighborhood of a^ associated with the entry set [q. z2] as a set of input symbols j\f(k. zi. z 2) =f {ojt*,„|z7i = di(z’i. /) where 0 < / < q , z 2 < / < Ls} (4.3) The concept of a neighborhood is illustrated as the collection of symbols denoted by black dots in Fig.4.3. The first index q is called the pivot entrg of this entry set and the set itself is called the pivot entry set. Notice that the neighborhood J\f(k. ii, z2) can take on one of values in the space . Specifically, when the entry set has only one element z. the neighborhood of a* associated with it is just j\r(k. i) =f .V *(A :, z, z) with M L“ possible values. Decision-feedback equalization (DFE) [BePa79] is a generic solution for ISI m it igation. For the S-ISI channels, both the linear feed-forward and feedback filter in a conventional DFE can be fairly long. In [FeGeFi98]. a fast algorithm is de veloped to compute optim al DFE settings for the S-ISI channels. Although the ‘Due to its nice structure, the performance of the regular S-ISI channel is analyzed in Section 5.5. 65 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 1 1 28 22 21 17 10 0 p(i) 5 4 3 2 1 0 / a sparse ISI channel input sequence - < ak „ * AT(k.2)* a > c Figure 4.3: Neighborhood of a*, for a sparse ISI channel. complexity is relatively low. a DFE-based detector may suffer a serious perfor mance degradation for severe S-ISI channels due to the error propagation effect. Recently, some efforts have been made to adapt the YA to the S-ISI channels. In [BeLuMa93. BeSa94. BeMa96], a so-called m ulti-trellis YA (MYA) is developed. Compared to the full YA. the MYA saves significant complexity. However, its de velopment is based on the construction of an irregular trellis. This construction is ad hoc due to om itting many correlated symbols and making some instant de cisions approximately. W hen the S-ISI channel has any two adjacent strong taps, this construction is forced to neglect strong symbol correlations. This can result in serious performance degradation. Furthermore, when the structure of the S-ISI channel is complicated {e.g., >5 non-zero taps) and a reasonable decision depth is used (usually 5L - 7L [HeJaTl]), the construction itself becomes fairly complicated. In [McKeHo98], the parallel trellis YA (PTYA) is developed for the simple S-ISI channels (called zero-pad channel class in [McKeHo98]). Due to the regular struc ture of a simple S-ISI, it can be shown that the PTYA yields the MLSE as the YA does. Consequently, the PTYA can only be applied to simple S-ISI channels. Both the PTYA and MYA have a complexity determ ined approximately by the number of non-zero taps instead of the length of memory as in the YA or SISO algorithm . 66 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4.3.2 T he Sparse SISO A lgorith m 4.3.2.1 A D istributed SISO A ssociated w ith a Single E ntry Associated with a single entry of an S-ISI channel, e.g., the f-th one. we can derive the following formula m athem atically: Pi.e(a*) =f Cl P r(;t+p(j)|afc ) = Cl Y , Pr(2*+p(i)M M ),a fc ) [ I P r(a,) (4.4) A ' a j € A ( f c . i ) Pi(afc) =f c2 Pr [rfc+p(i), a*] = c2 Pi.e(ajfc) Pr(afc ) (4.5) where the conditional probability P r ( ^ +p(i) |A/*(fc. i), a*) is com pletely determined by the channel noise model {e.g., it is a exponential function when the noise is AWGX). and C[.c 2 are normalization constants. The operation defined in (4.4) has the soft information about input symbols as both its input and output. Therefore, an iteration can be realized by feeding the output back to its own input port. This procedure is called self-iteration. Denote the quantity P(-) after the n-th iteration by P (n)(-). then the self-iteration suggests an iterative SISO algorithm : \ r(k.i)eAL‘ < ij e.'.'(k.i) P!"+ ,,(a*) = c, Pj-,e+1,(afc )P-0)(afc) (4.7) where n > 0 and P j°'(at) = Pr(afc). Obviously, this is a SISO algorithm . W ith little effort it can be observed that this SISO algorithm is based on the same operation principle as the 2D- 1 in Section 3.3. This is because a similar Bayesian network model in Fig.4.4(a) can be established for a ID ISI channel. This Bayesian network is not cycle-free either {e.g.. the dashed polygon in Fig.4.4(a)). It should be noted that there exists a fundam ental difference between the ID and 2D cases. Dem onstrated in [IvsFr98]. by adding nodes of the transition defined by the ID FSM. a cycle-free Bayesian network model can be obtained. However, no such m odification has been found for 2D cases. Same as in the 2D case, corresponding to this specific graphic model the fully connected BPA shown in Fig.4.4(b) is fairly com plicated. By picking a single pivot entry, a simplified BPA algorithm (Fig. 4.4(c)) can be obtained, which 67 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. input a} a-k- 1 afc- 1 ak ak^ t ak^ 2 P (aj) P (aj) o o o o o <J <J qi <J c* 9 9 9 9 9 # » 5 5 6 6 5 * » » » » A -2A-1 - C f c -T ic -I-T fc -2 ‘ * * * ‘ * * * ‘ » 6 0 6 6 0 0 0 6 6 6 O O O O O - * . - 2 * - 1 =k ^ 2 P r ^ x , ) p r ( =J|x,) observation Zj (a) (b) (c) 0 soft information processor - soft input of processor ► soft output of processor Figure 4.4: A Bayesian network model for a sparse ISI channel. is exactly the SISO algorithm derived above. Like the 2D4 algorithm , this SISO algorithm has a distributed architecture. Therefore, we will refer to this probability- based algorithm as the distributed APP-SISO algorithm (APP-DSISO). Its MSM version can be readily obtained via the translation rule. Unlike the 2D case in which no natural index order exists, this distributed architecture is enabled for the ID case by completely ignoring the tem poral order originally embedded in the ID ISI channel. 4.3.2.2 SISO for a Grouped Entry Set Although the DSISO algorithm is applicable to any single entry even if it is located in a grouped set (i.e., one could use the DSISO on a non-sparse ISI channel), a SISO algorithm developed for a grouped set can more fully exploit the channel structure. It will be shown that the DSISO is just an approxim ation of the following SISO algorithm . Given a grouped entry set [U, 2 - 2 = U + Lg\. define a state as s* =f {ai;-L3- ■ • • • o.k-1} with A.L g possible values and a transition as tk ( • * * .- • Sfc-ui)- Based on the definition of this state, a trellis can be built and the following probability- based quantities in (n - + - l)-th (n > 0) iteration are defined recursively as: &[n)(i) (4.8) i:tk=(S,,Sj) 4"'(< )= E (4.9) j:tk =(S, ,Sj) 68 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. with initial values of Qo^(i) = P r(s0 = 5,-) and P ^ \ j ) = Pr(s/c = Sj). For example, if the starting state is S,, then a ( 0 n'(i) = l ,a 0 (j) = O.j ^ i. If no knowledge about *o is available, then a ^ ^ i ) = M ~L*. A combining operation similar to the one in (4.6) is used to calculate the quantity in both the forward and backward recursions: ' i r°U -j) = Pr(a* = A (i,j)) Pr(zk+ p(ll)\tk = (St, Sj). .\r(k. q. U)) X(.k,ii,i2)€A Ls~ L9 x n p !rh « j) (-i-io) where A (i.j) is the value of the input symbol driving the forward transition 5, — > Sj. After finishing both recursions, two types of soft output can be obtained by p S y V * = - - w = £ 4 " ’0 ' ) 4 ”’u ) (-i-ii) Sj * O f c =« 4 n i P £ j l)(a* = A m) = c P\*+l)(ak = A m)/P r(a k = A m) (4.12) where c is a normalization constant and P|°e(aA.) = Pr(a*). The latter quantity is an "extrinsic" information and should be fed back if further iteration is required, and the former one can be used to make hard decisions after the final iteration. Again, this APP-SISO is suboptim al due to the ad-hoc operation in (4.10) and self-iteration is required to improve the performance. When ix = i-> , Lg = 0 and S = 0. the recursions in (4.8)-(4.9) and the completion steps in (4.11)— (4.12) are degraded to (4.6)-(4.7). i.e.. the APP-DSISO algorithm . Therefore, contrasted with the DSISO. this algorithm will be referred to as the "grouped" SISO algorithm (GSISO) associated with the given entry set. As with the DSISO, the GSISO is also built on a Bayesian network model for the ISI channel (e.g.. the upper one in Fig.4.5). Unlike the one in Fig.4.4. this Bayesian network model does not ignore the temporal axis completely because of using the memory nodes tk = (ak_ i,a k). Consequently, the resulting simplified BPA - GSISO is a tem porally recursive algorithm. It should be note that by defining tk = a£_3, a cycle-free Bayesian network (the lower one in Fig.4.5) can be obtained, on which the optimal BPAs (i.e.. the VA and the FI-SISO) can be built. Thus, our approach can be viewed as achieving a complexity reduction 69 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. < u - f f * f » f f ► -► 4 »• ^ ^ ^ ^ ^ v» - T f c T C f c J ’ ' ' | < f c = (at.m-!) ISI ¥ • * * * ¥ * = ( “ fc-“ fc-i • « i - 3 ) Figure 4.5: Bayesian network models for an ISI channel. by reformulating a cycle-free Bayesian network by an equivalent Bayesian network with cycles, then applying the em ad hoc iterative BPA. Also, the MSM version of The GSISO can be readily obtained. Since both the DSISO and the GSISO are developed for the S-ISI channel, we will call them together the sparse SISO (S-SISO) algorithm. 4.3.2.3 D ecision Feedback SISO and M ulti-SISO A lgorithm s I1 1 the DSISO or the GSISO, the soft inform ation combining is conducted over the whole neighborhood /) or respectively. This com bining operation determines the complexity of the S-SISO algorithm s. In [ChChXe98, ChCh99b], the decision feedback technique has been successfully used to simplify the SISO algorithms. This technique is also applicable to the DSISO and GSISO. Define /[. /■ > ). the truncated neighborhood, as a subset of z1 ? i2). and denote its complementary set with respect to J\f(k. ii. i2) as N f{k . z1 ; z '2). Then M {k. il: i2) is obtained by assigning the entries in Mtc(k, z ‘i, io) with corresponding tentative hard decisions. The set J\ff(k, z'i, z ’ o), which has L j elements, is called the feedback entry set. There are many ways to choose i\, z '2), e.g., the entries with relatively small 70 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. values can be excluded. By this complexity reduction approach, we can approximate (4.6) and (4.10) as r ! : r ' ! (a * ) = C, 5 3 n P (4.13) \'<{k.i)<ZAL , ~ Lf aJ £ A ' j ( A : . i ) P r(cfc+p(il)| tk = ( S ^ S j h t f i k .i L h ^ A r i k .h .i z ) ) \' d k ,i u i2)e A L3~ L3~ Lf * n Pr(ac = --l(i-j)) ■ (4.14) Oj 6 A t ( t.i i . I ; ) respectively. The resulting algorithm will be referred to as the decision feedback (DF) S-SISO algorithm. Given Lf. the complexity of fusion processor in either the DSISO or GSISO is reduced M Lf times. The tem porary hard decisions can be obtained by thresholding the soft output of the last self-iteration. It has been shown in [ChChXe98. ChCh99b] th a t the self-iteration would significantly compensate the performance degradation caused by the decision feedback. On the other hand, either the DSISO or GSISO can be executed for different pivot entries or entry sets (if there are several grouped entry sets) simultaneously. For example, if an S-ISI channel has two m ajor entries p(2) = 5 and p(5) = 43. Then two DSISOs for i = 2 and i = 5 can be run in parallel with :V 2 and .V g self-iterations separately. In general, if a DSISO algorithm is run for each entry index in set Z < / and a GSISO algorithm for each pivot entry index in set I g. the final soft output for a probability-based algorithm can be obtained as: P (ak = A m) = P (0)(af c = .4m) P £ l)(ak = .4m) P ^ f a * = A m) (4.15) i e l j j £ l g The above algorithm will be referred to as the multi-SISO algorithm since it con sists of several SISO submodules running in parallel. Intuitively, the multi-SISO algorithm should yield a more reliable sequence estim ate than its submodules since it exploits more structure inform ation about the S-ISI. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4.3.2.4 Com parison o f Features and C om plexity An index of the com putational complexity C can be defined as C =f T N r where T is the number of transitions. :V the num ber of self-iterations and r the number of recursions. For example, for the YA. C(YA) = M L+l x 1 x 1 = M LJrX. In Table 4.1. we tabulate T. N. r and C for several algorithms that are applicable to an S-ISI channel. Some im portant features of these algorithms are also listed. W ith the VA full SISO PTVA L MVA DSISO GSISO Applicable S-ISI arbitrary arbitrary sim ple short arbitrary (single) arbitrary (grouped) O ptim ality opt opt opt sub-opt sub-opt sub-opt In/O ut s /h s/s(h ) s /h s /h s /s(h ) s/s(h ) T A/ L t 1 M L+l M L* + l ~ M Ls+l M L’ + l N 1 1 1 1 N N r 1 2 I 1 1 2 c m l * 1 2 A/ L + 1 M l ’ + 1 ~ M L*+l N M L‘ + l 2 N M L- + l Table 4.1: Comparison of algorithm s for S-ISI channels. Note "s" stands for soft and ”h" for hard. most flexible applicability and highest complexity, both the YA and the full SISO algorithm yield the optim al sequence estim ate. However, it is obvious th at they are infeasible in practice even for an S-ISI channel with binary inputs (M = 2) since L 3> 1. On the other hand, both the PTYA and the MYA have much lower complexity (Ls L) and acceptable performance, but their applicability is seriously restricted. Nevertheless, the S-SISO algorithm s are com putationally sim pler than the YA and full SISO while more applicable than the PTYA and MYA. Furthermore, the flexible architecture of S-SISOs enables many ways to reduce the complexity {e.g., the DF- SISO). increase the applicability (e.g.. the multi-SISO), and improve the performance (e.g.. self-iteration). Not listed in the table, a DF-SISO has the same features as its full version counterpart while its complexity index is M Lf times smaller. Also, a multi-SISO has the same features but with C = M 1’ * 1 (^2ield Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4.3.3 D esign R u les o f th e Sparse SISO A lgorithm s When an S-ISI is given, the diversity of the S-SISO algorithms makes the design of an S-SISO based detector difficult. In this section, several rules will be established numerically to ease the detector design. In the following numerical experiments, we always assume th at the transm itter uses the BPSK signaling scheme {i.e., ak = dz\/Eb). and the output of the S-ISI channel is corrupted by an AWGX n k with E { nl} = -Vo/‘ -- All S-ISI channels are normalized {i.e., ||/ ||2 = 1) to preserve the SXR. The S-ISI channels which will be used are listed in Table 4.2. Moreover, the simulation result will be labeled by (the algorithm used)2-(th e channel sim ulated)-(the pivot entry(th e size o f the pivot entry set if > l))-(th e number of iterations used). W ithout loss of generality, the MSM version of a given S-SISO algorithm is used in experiments. i 0 1 2 3 4 5 A P{i) 0 13 14 20 27 / f(i) 0.32 -0.25 -0.12 0.3 0.17 / B P{i) 0 4 10 11 17 21 f{i) 0.72 -0.64 -0.85 -0.52 1.3 0.67 C P{i ) 0 6 12 18 24 / f d ) 0.29 0.5 0.59 0.5 0.29 / D p(0 0 7 14 21 28 / f{i) 0.2 -0.4 1.0 0.6 -0.3 / F. P{i) 0 4 5 / / / f { i ) 0.22 0.41 0.29 / / / G p{ 0 0 10 26 27 89 103 f{i) 0.36 -0.24 0.38 1.0 -0.23 1.19 Table 4.2: S-ISI channels used in numerical experiments 4.3.3.1 The P ivot Entry and the Type o f SISO To apply the S-SISO for an S-ISI channel, the pivot entry (set) must be assigned first, which in turn determines the type of S-SISO to be used. The choice of the pivot entry (set) is dependent on the structure of the S-ISI channel. Channel A J Usually, we use the initial letter of the name to represent the algorithm. 73 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. is defined in Table 4.2, and Channel A* (0 < i < 4) is obtained by replacing the value of i-th entry by 0.8. Obviously, the entry with value 0.8, i.e., the t'-th entry for Channel A,, dominates the channel in power. In Fig.4.6, the BER results of S-SISOs are plotted versus SNR =f 2Eb/NQ . The BER curve for a channel without ISI is also included as the base line of performance. In Fig.4.6(a), a DSISO is run for Channel A3 with all possible choices of the pivot entry. The results clearly show that the detector using the dominant 3rd entry significantly outperform s other detectors. Therefore, when an S-ISI channel has a dom inant entry, it should be chosen as the pivot entry- for a DSISO. Also note th a t the ranking of curves corresponds to the order of the m agnitude of the pivot entries used. The curves in Fig.4.6(b) further support this conclusion. No m atter where the dom inant entry is located in the channel, the DSISO associated with it always performs near-optim ally (~ ld B away from the No-ISI performance at a BER of 10-4). In particular, for Channel A2. the entry set {1.2} is the dominant one. It can be expected th a t a GSISO associated with this set outperforms a DSISO associated only with the 2nd entry. Therefore, when a grouped entry set dominates an S-ISI channel, a b etter choice is to use it as the pivot entry set for a GSISO. In general, when a detector can only use a single SISO. the entry (set) with the most energy (may not be dom inant) should be chosen as the pivot entry (set) for the DSISO (GSISO). 4.3.3.2 C onvergence R ate o f Self-Iteration Because of the approximated inform ation combining used in the S-SISO. self-iteration is employed for a better combining effect. Generally, the more self-iterations are used, the better the performance is. However, as discussed in Section 4.3.2.4. the complexity of the algorithm is proportional to the num ber of self-iterations used, .V. Actually, the BER performance of a S-SISO converges after certain number of self-iterations. Therefore, a minimum N exists which yields effectively the best per formance with minimum complexity. We denote this em pirically optim al N as A'opt • For a given S-ISI channel, Ar opt is determ ined by both the position and value of the pivot entry used. The BER convergence curves are plotted in Fig.4.7 for Channel B (see Table 4.2). It is clearly shown by Fig.4.7(a) th at the closer to L/2 (i.e., the 74 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. D-A3 -0-13 N D-A3-I-I6 j D-A3-2-9 |_ _ D-A3-3-5 | D-Aa-4-14 | 10"\; 30 1 0 ° E DFE(28.27)-A2 3 s D-Aq -O -o D-Ai-l-5 '■ © ------D-Ao-2- i D-A3 -3-5 D-A^— l- 6 &— G-At- 1-0 No ISI 6 9 12 ■2Eb/ X 0 (dB) Figure 4.6: Performance of an S-SISO with different pivot entries, (a) select different pivot entris for Channel A3. (b) select dom inant entry (set) as the pivot entry. center of the S-ISI channel) p(i) for the pivot entry i is. the smaller .Vopl. Specif ically. the DSISO associated with the 2nd or 3rd entry has the same convergence rate as the GSISO associated with the entry set {2.3}. The similar trend can be observed in Fig.4.6(a) where A'opt is used for each curve. However, when the pivot entry (set) dominates an S-ISI channel, the jV opt becomes alm ost independent on its position. This is shown in Fig.4.7(b). Although the entry 1 is much closer to the channel center than the dom inant entry 0. D-A0-l has an iVopt 3 times as large as D-A0-0 does. In all these cases, the BER curves become very stable after Ar opt self-iterations. 1 o Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. (a) 20 40 60 80 100 Number of Iterations 120 O . n A n n M O rlp N i O - D-.A q-I 23dB) [ ---- 3---- D-Ai-1 (l2dB) ! ---- *---- D-Aa-2 (13dB i o • G-A>-1 (12dB) 1 v n \i * } M ' V I m ‘ 'C O . ; L O . n c ■ m - \j- ,* \3 - j ( liGD) ‘ H A, t M ^ r ] P ^ 1 e- o 2 0 a - o - e- © 10 15 20 Number of Iterations Figure 4.7: Convergence property of the sparse SISO algorithms. The num ber using dB-unit is the 2£, 6/A’ 0(dB) at which the corresponding algorithm is sim ulated. The number attached to each curve is the corresponding -V oPf (a) Using different pivot entries for Channel B. (b) Using dom inant entry for Channel A,. 4.3.3.3 Impact of D ecision Feedback For an S-ISI channel, when some of the entries are relatively small, a detector can employ the DF-SISO to reduce the complexity efficiently. However, as a side effect of any complexity reduction technique, the detector will suffer certain performance degradation. Using {1}. {3} or {1.3} as the DF entry set. DF-DSISO algorithm s as sociated with entry 4 are tested for Channel B. Fig.4.8(a) shows that the convergence rate of a DF-DSISO is significantly slowed down due to the feedback. Specifically, the more tap power the DF entry set has, the slower the DF-SISO converges. By 76 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 10 D-B-4 ( 16dB) D F (l)-B -4 (16dB) DF(3)-B-4 (16dB) DF(13)-B-4 (16dB) s -100 70— & - no-* (a) 20 40 60 80 Number of Iterations 100 12C 10“ > 10' D F(13)-B-4-80 ( 0 . 4 4 ) 3 DF(13)-B-4-110 (0.6) i J 10 L D-B-4-45 (1.0) EDF(3)-B-4-70 (0.78) 10‘4 gDF(3)-B-4-45 (0.5) 10 DF(l)-B-4-100 (1.1) DF(l)-B-4-65 (0.72)- (b ) 10 12 14 16 2£T 6 /.Vo (dB) 18 20 Figure 4.8: Impact of hard decision feedback. The num ber following "DF-" is the decision feedback entry set. (a) convergence property of DF-SISOs. (b) BER perfor mance of DF-SISOs. The number following a label is the corresponding complexity index normalized to that of D-B-4-45. using their own .V O P f these DF-DSISO algorithm s only suffer a small performance degradation (< ldB. see Fig.4.8(b)). It implies th at self-iteration greatly compen sates the effect of DF. Also, the more tap power the DF entry set has, the more performance loss the DF-DSISO suffers. Consequently, the total complexity saving is less than M Lf due to the increase of Ar opt. Sometimes, a DF-DSISO using its own Aopt (e-9-• DF(l)-B-4-100) is even more complex than the corresponding DSISO. By changing .V used, we may make a tradeoff between the complexity and the perfor mance. For each DF-DSISO, an N < Nopt is properly assigned and the performance 11 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. is shown in Fig.4.8(b). By suffering about another ldB perform ance loss, complexity has been saved. It should be expected that a larger feedback entry set may cause the DF-SISO to fail due to the serious error propagation. 4.3.3.4 Perform ance Im provem ent by M ulti-SISO When an S-ISI does not have any dominant entry (set), any individual DSISO or GSISO may not provide a satisfactory performance. In this case. multi-SISO can be employed to get a better performance. Channel C (Table 4.2) is a simple but severe S-ISI channel. Fig.4.9 shows the performance of several multi-DSISO algorithms for it. First we observe th at the more submodules a multi-SISO has. the larger .Vopt >D-C-2-S M-C-12-14 16 IS 20 22 24 26 28 30 32 2£fc/.Y0 (dB) Figure 4.9: Performance improvement by using the multi-DSISO. The notation "M- C-12-14" means that this multi-SISO consists of two DSISO submodules that asso ciated with entry 1 and 2 separately. is. This trend is expected since these submodules are run independently, the multi- SISO will not converge until all its submodules have converged. Therefore, Ar o r> t increases when more subm odules are included in the multi-SISO. Compared to the D-C-2-8. the M-C-12-14 improves the performance by 3.2dB in SXR at a cost of 3.5 times complexity increase, and the M-C-123-18 obtains a 5.3dB performance gain by increasing the complexity 6.75 times. 78 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 4.3.3.5 Sum m ary o f D esign Rules In summary, when an S-ISI channel is given, one can design an S-SISO based detector by the following rules: 1. If the S-SISO channel has a dominant entry (set), choose it as the pivot entry (set). Go to step 4. 2. If a single S-SISO will be used, select the dom inant or heaviest entry (set) as the pivot entry (set). In this case, if two entries (sets) have similar tap powers, choose the one closer to the channel center as the pivot entry (set). Go to step 4. 3. If the S-ISI channel does not have a dom inant entry (set), a multi-SISO is suggested. Choose the heaviest entries and entry sets as the pivot entries (entry sets). 4. If the S-ISI channel has some fairly small entries, choose them as the DF set and use the corresponding DF S-SISOs for each pivot entry (set). Otherwise, use the corresponding S-SISOs for each pivot entry (set). 5. Find -Y op[ numerically. Use it or choose a smaller .V for complexity saving. 4.3.4 B E R P erform ance of the Sparse SISO s In this section, numerical experiments are conducted to evaluate the performance of the DSISO and GSISO. The results are compared to the performance of the DFE. MYA and PTYA when they are applicable. The DFE(A’/./\(,) has a linear feed forward filter of length K f and a feedback filter of length A'& . The MMSE criterion [SmBe97] is used in finding the filter coefficients. The input of the feedback filter is the previously detected symbols. First, a simple S-ISI channel - Channel D (see Table 4.2) is tested, to which the PTYA (also MYA) is applicable. The simulation results are shown in Fig.4.10. Compared to the PTYA th at yields the optimal performance (i.e., MLSE). a DSISO assoicated with entry 2 performs almost 2dB worse while its complexity is 6 times greater than that of the PTYA. This situation would become even worse for the 79 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 10 _ D-D-2-6 ■ 3 10 D FE(29.2S)-D ,-i 10 PTVA-D (MLSE)J 0 10 15 o 2Eb/N 0 (dB) Figure 4.10: Simulation results for simple S-ISI Channel D. more severe simple S-ISI channels. Hence, the S-SISO should not be considered for a simple S-ISI channel. Channel E (see Table 4.2, refer to channel 2 in [BeSa94]) is an S-ISI channel with only 3 entries to which the MYA is applicable but not the PTYA. the sim ulation result (8-PSK data) about the MYA in [BeSa94] is duplicated in Fig.4.11. It is clearly shown th at the multi-DSISO algorithm M-E-12-16 performs about 0.5dB better than the MYA at high SNR. At the same tim e, the DFE(11,4) performs more than 5.5dB worse than the multi-DSISO. Since the best achievable BER performance for this channel must be to the right of the BER curve for the NO ISI channel, it is clearly shown th at the S-SISO (i.e.. M-E-12-16) performs very close (< ldB )to the MLSE. Two severe S-ISI channels - Channel F and G - are used to evaluate the S-SISO algorithms. Channel F is the S-ISI channel is defined with / 0 = f 2 = fio = f n = 0.0993. / 1 = foo = f 22 = 0.352 and f 2i = 1.0. By the approach discussed in Section 5.5. d'm in for the corresponding 2D ISI/AW GN channel is found to be 0.76 [Ch96]. Compared to the NO ISI channel, this regular S-ISI causes a performance loss of over 2.38 in SNR. This gives the MLSE lower bound shown in Fig.4.12. By observing the structure of Channel F. we find th at the entry set {20.21,22} is grouped and has most of the tap power. Following the design rules in Section 4.3.3.5, the best choice should be a GSISO associated with these 3 entries, for which the simulation 80 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. I 10' 10' _ D F E (11.4)-E 3 No ISI 10' 1 0 " * M-E-12-16 10" ’ 20 16 IS oo 24 26 14 2 E ,/X 0 (dB) Figure 4.11: Sim ulation results for S-ISI Channel E (8-PSK). result is presented in Fig.4.12. Fig.4.12(a) shows the B E R performance of G-F-3 using different iteration num ber N . Clearly, a huge perform ance gain is achieved through self-iteration. Relatively, the first 4 iterations give a much larger SXR gain than the following 3 iterations which only give about ld B SXR improvement. The inset in Fig.4.12(a) also shows th a t the convergence speed is much larger at the first 4 iterations than in the following iterations. Fig.4.12(b) shows th at the G-F-3-7 per forms only 1.2dB away from the MLSE lower bound. In com parsion, a DFE(43.42) performs more than 2.1dB worse than the GSISO. Moreover, if necessary, the GSISO can provide the soft information for further information processing but not the DFE. Channel G is a farily long S-ISI channel as defined in Table 4.2. Totally. 6 non zero taps spread over a memory lenght of 104. In this case, any standard SISO algorithm is inapplicable. By obervation. we find that Channel G is dominated by two entries - 3 and 5. Based on the design rules, the reasonable choices are a D-SISO assoicated with 3 or 5, or a multi-DSISO associated with 3 and 5. The simulation results are shown in Fig.4.13. Sim ilar to the G-SISO, it is shown by Fig.4.13(a) that the DSISO gets a huge SXR gain through self-iteration. Also, the relative SXR gain becomes smaller when the iteration number gets closer to A'opt = 10. By using these curves, one could make a tradeoff between the performance and complexity. However, relative complexity reduction obtained by using N < :V opt 81 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. G-F-3-A* — i F 5" I - 4 r r- i<5-C Xo of Iterations opt — 7 10 8 6 10 18 12 14 16 ( a ) 2E,JN0 (dB) DFE(43.42)-F MLSE ^ lower bound G-F-3-7 " L C f‘ L , 6 8 10 16 IS 14 Figure 4.12: Simulation results for S-ISI Channel F. (a) BER perform ance of G-F-3 w ith differnt iteration num ber N . The inset is the convergence curve at a SXR of 16dB. (b) BER perform ance of G-F-3-7. may not be significant. For example, using .V = 5 is as half complex as using X'opt = 10 at an SXR cost of 9dB. Usually, using a decision feedback S-SISO is a b etter choice for saving the complexity. The performance of different schemes are shown in Fig.4.13(b). Since Channel G is a quite long S-ISI channel without a regular structure, it is hard to analyze it. Alternatively, we list the curve for Xo-ISI channel as a baseline for comparison. For Channel G, we design a D FE with a 104- tap feed-forward filter and a 103-tap feedback filter. Its perform ance for Channel G is also presented in Fig.4.13(b). Using 12 iterations, the D-G-3 works as well as the DFE(104.103). In assoication with the entry 5 th at has more power than the 82 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 10 I 10 o. 10' * o . X . o.. 3 10' D-G-5-.Y 10' =10 I O '* 30 10 25 o ! 0° w i 10' D-G-3-12 •) 10 M-G-35-1 •3 10 D-G-5-10 DFEllOJ.lOSJ-G^ 10 No ISI 1 0 ' ’ 6 S 12 16 4 14 Figure 4.13: Sim ulation results for S-ISI Channel G. (a) BER performance of D-G-5 with differnt iteration number .V. (b) BER performance of S-SISO algorithms. entry 3. the D-G-5-10 improves the performance by about ldB in terms of SXR. It is expected th at a multi-SISO combining these two D-SISOs should achieve a better performance. Fig.4.13(b) clearly shows that M-G-35-10 performs 2dB better than D-G-5-10 and more than 3dB better than both D-G-3-12 and DFE(104.103). The BER curve for M-G-35-10 is only 1.5dB away from that of XO-ISI channel. It should be noted that at the low SXR. DF(104.103) outperforms two D-SISOs. 83 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. C hapter 5 Toy Problem s and O pen P roblem s 5.1 The Intractability of F inding th e M LPE 5.1.1 A B rief In trod u ction to th e C o m p lex ity T heory The theory of XP-completeness [GaJo79] is about the com putational complexity of decision problems. A decision problem is one whose solution is either "YES" or "NO". A decision problem II is said to belong to the class lVP (nondeterministic polynomial) if it can be solved by a nondeterministic Turing machine in polynomial time. A decision problem FI is NP-complete if even.- problem in XP can be trans formed to n in determ inistic polynomial time. A problem IT which is not necessary in XP. is said to be NP-hard if the existence of a determ inistic polynomial-time algorithm implies the existence of such an algorithm for every problem in XP. By the definition of the XP-completeness, to prove a single decision problem n XP-complete. all we need to do is to show that some already known XP-complete problem n 0 can be transform ed to II. More rigorously, the process of devising an XP- completeness proof for a decision problem II will consist of the following four steps [Ga.Jo79. pg 45]: (1) show th at II is in XP: (2) then select a known XP-complete problem IIo. and (3) construct a transformation / from n 0 to II such that (4) / is a polynomial-time transform ation. Usually, the problem of interest is not in the simple form of a decision problem. Therefore, we cannot apply the XP-completeness theory directly to such problems. However, in most cases, we can construct a decision problem closely related to the problem and show th a t the original problem is ”no easier" than the decision problem, namely, it is NP-hard. 84 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5.1.2 D ecision P ro b lem R elated to 2D D L M P roblem In the formal language, the 2D DLM (MLPE) problem in (3.4) can be described as: P r o b l e m : 2D DLM (2DLM). I n s t a n c e : Given an N x .V observation page Z and a cost function c(i.j) =f | — f(Uj) * b(i, j)\2- b(i.j) is an HD binary random vari able. Filter is a given, finite size. truly-2D linear filter. Q u e s t i o n : W hat is the input data m atrix B which minimizes the page cost H («j)C (L j)? The concept ~truly-2D" means that the non-zero taps in the coefficient m atrix are not located on a single straight "line”. For example, a filter defined by ( 1 0 0 \ 0 1 0 \° 0 1 / (5.1) is not a "truly-2D” filter since all non-zero taps are located on the diagonal "line” (i.e.. a ID ISI along the diagonal direction). It is easy to observe that this specific 2D DLM problem can be solved optimally by the YA by slicing the observation along the diagonal direction. W hat we need to do is to transform this optim ization problem into a related decision problem which can be shown to be XP-complete. We are then able to conclude that the problem 2DLM is XP-hard. The related decision problem can be stated as: P r o b l e m : 2D Threshold Decision (2DTD). I n s t a n c e : Given an N x .V observation page Z and a cost function c(i-j) =f l~(LJ) — f ( i - j ) * b ( i.j )|2. b(i.j) is an IID binary random vari able. Filter f ( i , j ) is a given, finite size. truly-2D linear filter. Let T be a positive constant. Q u e s t i o n : Does there exist an input d ata m atrix B such that the page cost £(.../) c(f- i) < r ? 85 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5.1.3 T he N P -C o m p leten ess o f 2D T D T h e o re m 1 The decision problem 2DTD is NP-complete. Proof: It is easy to see that 2DTD G XP since a nondeterm inistic algorithm need only guess a input page B and check in polynomial time whether the corresponding page cost is sm aller than T. In [Co90] Cooper showed that the probabilistic inference using Bayesian networks is XP-hard. Fig.3.17 shows a Bayesian network (see next section for more discus sions) embedded in the 2D ISI/AW GX detection problem. The calculation of the a posteriori probability of an input page Pr(B jZ ) is an instance of the probabilis tic inference on this Bayesian network. Cooper's conclusion says th at the decision problem that whether there exists an input page such th at P r(B |Z ) > c is XP- complete. R estating this probabilistic inference problem in the m etric domain, we obtain the 2DTD problem with the threshold I be a function of c. Obviously, this transform can be accomplished in polynomial-time (with respect to the problem size .V). Hence. 2DTD is XP-complete. ■ By this result, we can conclude th at the MLPE problem is X P-hard. Furthermore in [DaLu93] Dagum and Luby showed that certain approxim ating probabilistic infer ence in Bayesian belief networks is XP-hard. It directly implies th at certain efforts to solve the M LPE problem approxim ately are almost helpless. 5.1.4 On th e 2D index The complexity associated with the MLPE mainly comes from the lack of a natural and meaningful {e.g.. tem poral) order in the 2D index. Although the ID MLSF. problem can have a Bayesian network model as in Fig.4.4 the index of which is lack of (temporal) order, by grouping the input symbols into transitions as in Fig.4.5. a equivalent cycle-free lD-fashion (temporally ordered transition chain) Bayesian network is obtained, to which the optim al BPA is applicable [Pe86]. So, the MLSE problem is not X P-hard. However, the 2D case is significantly different. Treating each input bit as a node, we have a densely connected (due to the inference from the same observation) 2D network shown as the upper ones in Fig.5.1. We want to show that no m atter how we group the input bits into composite nodes, the resulting 8G Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 0 9 1 1 0 0 999 m tm t, 9 9 0 9 0 9 999 9 9 0 0 9 9 999 (a) (b) (c) • • • • • • • • Figure 5.1: Bayesian networks embedded in the 2D M LPE problem. Bayesian network is still connected in the 2D manner (i.e.. have cycles). First, the grouping scheme must be problem-size independent. For example, by grouping a whole row into a composite node, we get a ID-connected Bayesian network. However, the cardinality of these composite nodes itself grows exponentially with the problem size .V. Therefore, the resulting ”MLSE" problem is still X P-hard. W ithout loss of generality, we categorize the grouping schemes into 3 classes. The index set of the bits in a composite node is called the grouping mask. e.g.. the shaded regions in Fig.5.1. The first grouping scheme generates the composite node using a fixed ID mask (see Fig.5.1(a)). The resulting Bayesian network has the same connection structure as the original one except for the composite node. The second grouping scheme generates the composite node using a fixed 2D mask (see Fig.5.1 (b)). Again, the resulting Bayesian network has the same connection structure as the original one. The third grouping scheme generates the composite node using variable masks (see Fig.5.1(c)). Focus on a typical composite node {e.g., the one with the vertical ID mask in Fig.5.1(c)). it must be two dimensionally surrounded by other composite nodes since the grouping scheme must consider all the nodes in the original Bayesian network. Since the original network is 2D-connected. In the modified Bayesian network, this composite node must connect to the surrounding composite nodes in the 2D fashion. O ther grouping schemes may use grouping masks overlapping to 87 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. each other. In those cases, the connection would be more dense. In summary, we conclude that the em bedded Bayesian network for the M LPE problem cannot be simplified in the sense th at the 2D connection pattern results in m any cycles in the network. 5.2 Sym bol versus Sequence D etectio n The maximum likelihood bit(symbol) estimate (MLBE) arg max P r(af c = .4m,zJ*) (5.2) achieved by the APP-SISO in Section 2.1.2.2 and the maximum likelihood sequence estimate (MLSE) arg min M (af c = Am, z£;) (5.3) < U - = . 4 m achieved by both the YA [Fo73] and the MSM-SISO in Section 2.1.2.3 are two dif ferent but closely-related optim ization criteria [HaCoRi82]. W hen using BER as the performance measure, the MLBE can not be worse than the MLSE. Although it is quite interesting to find out how much the relative gain is. in some cases it can be shown that the MLSE is strictly equivalent to the MLBE. 5.2.1 H igh S N R C ase First, inspired by [ChHa66], we obtain: T h e o re m 2 (Im p ro v e d fro m [C hH a66]) I f there exists a sequence a f such that given the observations z[v P r(a f |z f ) > 0.5 (5.4) then the MLSE and the M LBE are equivalent. Proof. Because P r (a f |z f ) > 0.5, by the definition we m ust have a f be the MLSE. On the other hand, since Pr(a*|zr+ i) = £ P r (a f|zf+t) (5.5) 88 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. when ak = ak. the term P r(a(v|z f ) must be in the sum m ation. This implies that Pr(a\.|z{') > 0.5. which is true for 1 < k < K. By thresholding Pr(a<:|z(v). we obtain the MLBE. i.e.. a f . So. the MLSE and the MLBE are equivalent. ■ To clearly see the situation at high SNR. we need the following two Lemmas. L em m a 3 Assume the channel is an ISI/AW G N channel and the transmitted se quence is a f1. Given an arbitrary constant c : 0 < c < 1. then P r(P r(z f | a f ) > c) — ► 1 (5.6) as the S S R approaches infinity. Proof. From c < P r ( z f '|4 f ) = ( - ^ ) * e x p ( - £ * = ° ^ ). v 2tt(T 2.cr~ (5.7) we get K u d = ^ 2 wk < — 2a2(ln c + /\'ln (\/2 7 rcr)) (5.S) k=l where u is a chi-square random variable with K degrees of freedom. Specifically, when K is even we get P r(P r(z fr|a f ) > c) = Pr(u < — 2cr2(ln c -t- A' In( n / ^ t t c t ))) = l - e x p ( l n c + A - l n ( v ^ ff)) £ i ( - i m c + A - l n ( ^ ) ) ‘ := 0 — ^ 1 as o — ^ 0. (5-9) Fix the input signal power, a — ► 0 as SNR — ► oc. W hen K is odd. let v = u + w2 K+l. Then it is easy to show th at P r(u < — 2a2(Inc + A'ln(v/27r<r))) > P r(c < — 2cr2(ln c+ A' ln(\/27rer))) — > 1 as SNR — > oc. ■ 89 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. L e m m a 4 Assume the channel is an ISI/A W G N channel and the transmitted se quence is a f . Given an arbitrary constant c : 0 < c < 1 and an arbitrary sequence a* # a f . then P r(P r(zf |a* ) < c) — > 1 (5.10) as the SNR approaches infinity. Proof: From c > P r ( z f la f ) = ( ^ ^ ) y exp(— Wk)2). (5.11) \/2~a 2a- we get K u * T J > + wk )2 > —2ct2(In c + I\ ln{\/27To)) (5.12) k = l Since af1 a f . there exists at least one 5k / 0. Moreover, it is easy to verify that u is a noncentral chi-square random variable with K degrees of freedom. It is easy to show that given any e<C 1. P r ( |( 4 + wk )2 — Jj!| < e) — > • 1 as o — * • 0. Therefore Prfl-u - s2 1 < c) > Pr(Ui<Jt</v '{ |( 4 + u'k)2 ~ < 5 * 1 < e/A'}) K = I l P r( l ^ + < e/K ) -> 1 (5.13) k = 1 as cr — > 0. On the other hand, the right size of (5.12) becomes smaller than s2 — e soon when a — > 0. Therefore. P r(P r(z fr|a fr) < c) = P r(u > — 2a2(lnc-i- I\ ln(\/27rcr))) > P r(|u — s2| < e) — > 1 (5.14) when SXR — > oc. ■ Consequently, when the SXR is high enough, the conditional probability P r(a A \zK) = ______________P r(z * |a * ')P r(a f)______________ 1 ' ‘ ^ £ .* * * * P r(zM af') P r(a f) + P r ( z f '|a f ) P r(af') (0' i0j 90 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. should approach 1 with very high probability. Thus the condition th a t Pr( i f l z f ) > 0.5 could be satisfied most of the time. So. the MLSE and the MLBE should be identical and correct with very high probability at high SXR. It should be noted that in [LiYuSa95] the discussion about the asym ptotic optim ality (in the sense of the MLBE) of the metric-based soft inform ation packet is not rigorous since the arguments about the transm itted sequence are incomplete. 5.2.2 M -P SK Signal Case In ail .1/-PSK (M = 2l) signaling scheme, every /-long message bit sequence 6 [Z > 2 • • • h is mapped into an .1/-PSK signal s. Passing s through a complex AWGX channel, we observe At the receiver side, if the detector uses the minimum distance criterion, it optimally estim ate the message symbol-by-symbol (i.e.. /-bit by /-bit). This is actually the MLSE. We can also build a detector, which calculates Pr(Z>, = B\z) = YLs bt=B Pr(a l~)- to detect the message bit-by-bit. It is easy to verify that this is the MLBE. To get some conclusions on the relationship between the MLBE and the MLSE for the .1/-PSK Signaling scheme, we need the following lemma. L em m a 5 Given four points S1.s-2.s 3.s 4 in Fig.5.2 where a > 0 and b > 0. De note Pi(z) = exp(— cdr(z. s,)/2) with constant c > 0 where d(z.s) is the Euclidean distance between points z and s. (1) If y > 0, Pi(c) > P4(z). (2) If xy > 0. Pi(~)+P:i(-) > pn(z)+ p.i(~). And pl(z)+pz(z) = p-2(=)+Pi(z) if and only if xy = 0. • (x,y) S- b, # Sl —a o a «3 # - b Figure 5.2: A non-equally spaced QPSK constellation. 91 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Proof. (1) First, the farther Si is away from 2. the smaller Pi{z) is. It is easy to see from Fig.5.2 that when y > 0, must be closer to 2 than s4. Then p\{z) > p4(c). (2) Given 2. we have P i(2) +Pz(z) - M z ) - P4 (2) = Ci(ecby - e~cby)(ecax - e~cax) (5.16) where C [ = exp(— c/2(x2 + y~ -r a2 - + - b2)) is a constant. When x = 0 or y = 0. the left hand side (LHS) of (5.16) is zero. It is easy to show th at u — £ > 0 if tz > 1 and u — - < 0 if 0 < u < 1. So when x > 0 and y > 0. ecby > 1 and ecax > 1. the LHS of 1 1 (5.16) is positive. When x < 0 and y < 0. ecby < 1 and ecax < 1. the LHS of (5.16) is also positive. ■ By this lemma, we get T h e o re m 6 Given an AW G N channel, the M LBE is equivalent to the M LSE (i.e.. have the same decision regions) when using the naturally indexed BPSK and QPSK signaling schemes. Proof. By properly pairing the points in the constellations, the Lemma 5(1) tells that the decision region of = 0 (denote as DR(6i = 0)) must be the upper half plane for BPSK. QPSK and 8PSK. This is illustrated by the shaded regions in Fig.5.3. Then it is trivial to see that the MLSE is equivalent to the MLBE in BPSK since they have the same DRs. Lemma 5(2) directly implies DR(62 = 0) for QPSK is xy > 0. Rearranging 8PSK into two non-equally space QPSK sets. i.e.. (0.3.4.7) and (1.2.5.6). In both QPSK constellations we have the same DR(62 = 0), i.e.. xy > 0. Therefore, the 8PSK has xy > 0 as DR(62 = 0). Using the MLSE detection scheme in QPSK. DR(00) is just x > 0 and y > 0. In the MLBE detection scheme. DR(00) = DR(6[ = 0 and 62 = 0) = DR(6t = 0)f")DR(62 = 0) which is x > 0 and y > 0. Similarly. All DRs for other symbols are the same in both MLBE and MLSE. So, the MLSE is equivalent to the MLBE in QPSK. ■ For the 8PSK. although not shown analytically, we are able to find numerically that DR(& 3 = 0) is the shaded region shown in Fig.5.3. Therefore, the MLSE is equivalent to the MLBE in 8PSK as well. Xumerical experiments also showed that this is also true for naturally indexed 16-PSK constellation. By above results, we conjecture th at: 92 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 1 10 1 1 B P S K Q P S K SP S K (n a tu ra l index) Figure 5.3: Bit-by-bit decision regions in various PSK constellations. C o n je c tu re 1 Given an AW G N channel, the M LBE is equivalent to the MLSE when using the naturally indexed 2?-PSK (I > 1) constellation. We observed numerically th at the MLBE is not equivalent to the MLSE for SPSK when the constellation is not indexed properly. Specifically, for Gray indexed 2LpSK. we have the following fact: T h e o re m 7 The DR(bi — B m) is identical (modulo 2 -/2 ') to both the naturally indexed 2t-PSK constellation and the Gray indexed 2l-PSI\ constellation. Proof: An input bit sequence bib? • * - h is mapped to the f-th signal in the naturally indexed 2*-PSK constellation, where t = 2mbl_m. Fix 6, = 0. then signal indexes of all possible such input bit sequences assemble into blocks. Each block consists of 2l~l successive indexes and the distances between adjacent (on the circle) blocks are 2l~'. The Gray index can be obtained directly from the natural index via the mapping: b\b2 • • ■ b[ — > (b\ © b2)(b2 © 63) • • • (6/-1 © bt)b[ (5.17) 93 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. It is easy to check that this mapping is a one-to-one mapping. Just observe bit i and bit i + 1 in the natural index and the resulting z-th Gray index: 6i+l 0 ••• 0 1 ••• 1 0 ••• 0 1 1 bi 0 ••• 0 0 0 1 ••• 1 1 ••• 1 (5.18) 0 • • • 0 1 • • • 1 1 • • • 1 0 • • • 0 Since the lower (I — z)-bit in this Gray index, i.e.. (bi+t © b ,-+2) - - - (6/— i © b[)bt is just the 2i-,-ary Gray index, which is a perm utation mapping of the 2/_t-ary natural index. So it is easy to verify that fix 6, ©&,-_ i = 0 Gray signal indexes of all possible such input bit sequences also assemble into blocks each of which consists of 2l~l indexes and the distances between adjacent (on the circle) blocks are 2l~l. By the rotation sym m etry of the constellation, this two constellations must have the same decision region for their z-th bit except for a 2 t t / 2‘ angular translate. ■ 5.3 On M SM SISO A lgorithm s 5.3.1 N orm alization in M SM SISO A lgorithm s Although theoretically the only informative part of a soft information is the relative reliability measure among the different values which the quantity investigated can take, it is very im portant to normalize a soft information packet to ensure that the SISO is numerically stable. In the APP-SISO algorithm, the normalization is naturally included since a probability mass function has a total mass of 1. Usually, the metric for the MSM-SISO is defined as: M(d) = - Clln (P (rf))+ c 2 (5.19) where > 0 and C o are constants that may be selected to simplify the calculations. Specifically, in the case of AWGN with a double-sided density of A'o/2. let cx = .V0 and c2 = — .Vo/2 In(TrAo), we obtain M;(x/t) = \zk — Xk\2. Theoretically, the final results (i.e.. relative reliability measure) will not be changed by adding a constant to a given metric-based soft information packet (we will call it metric packet). First the information carried by this very metric packet has not been changed by adding this 94 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. constant. Then, the m arginalization and combining operations (i.e., min-sum) in a MSM-SISO algorithm will autom atically add the same constant to all the following related m etric packets. This will not change those m etric packets either. This property is w hat the normalization operation in an MSM-SISO algorithm is based on. In our experience, whenever a m etric packet about quantity d is obtained, we can normalize it via: M(d = Dm) <- M(d = D m) - min M(d = D m) (5.20) m so that the m inim um term in the m etric packet is forced to be zero. This valid normalization operation ensured the numerical stability in all our simulations. 5.3.2 F actor No in M SM SISO A lgorithm s Since the same (up to an additive constant) metric definition should be applied to all the metric quantities, when we calculate we also need the knowledge about the noise, i.e.. Ao- By choosing c2 = 0 (think of it as a norm alization operation), we have M[0)(a* = A m) = -A o ln(Pr(a* = .4m)) (5.21) It is interesting to observe that when the input symbols are IID. Pr(ajt = .4m) = 1 / |-A j and M -[0)(a A .- = .4m) = .V0 ln(|A j). By adding the constant — .V0 ln(|.Aj) to all these metric packets, we get M[°*(a*; = .4m) = 0. We also have M i(j^) = |cf c — x*..|2. Both of the soft inputs are independent on .V0. This proves the following theorem: T h e o re m 8 In the case o f AWGN, when the input symbols are IID, no knowledge about the noise variance is required for an MSM-SISO algorithm. On the other hand, when the input symbols are not IID, the knowledge about .V0 is necessary for im plem enting the MSM-SISO algorithms. 95 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5.4 On D istan ce Spectrum o f ISI C hannels 5.4.1 T w o Q uadratic Form s for th e D istan ce Given an ISI channel f = (/o / i • • • J lY and an error sequence between two input sequences, i.e.. e = (e0 ■ ■ ■ e*-)' =f — a2. We assume th at eo and e k are non-zero valued, and ek = 0 when k < 0 or k > K . The corresponding distance between the two input sequences is given by d e f d (e )= d (a 1.a ,) = | | / * e||, = K + L L \ £ £ > * - ' > a \ k=0 1 = 0 (5.22) / d e f U = 1) Toeplitz m atrix C o 0 0 ... o C l e0 0 ... o C-2 C l C o . . . o (5.23) \ £ k ~ l e K+L_i £k / Then the distance can be reformulated into a quadratic form of f as [Pr95]: d2(e) = (uf)'uf = f'(u'u)f = f'E f where E is an {L -I- 1) x (L + 1) real, symmetric and Toeplitz m atrix (5.24) E d e f / — U U = C o Cl c2 Cl C o Cl C l C o and C l c l-i c l - 2 \ Cl tL- 1 C/,-2 • • • C o / K-k C fc = eiei+k 1=0 (5.25) (5.26) 96 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Actually, it is easy to conclude from (5.24) th at E is positive definite. Alternatively, we can also define a (K + L + 1) x (A' + 1) Toeplitz m atrix d e f V = ( fo 0 0 ••• 0 \ f i fo 0 ••• 0 fz fx fo • • • 0 V 0 0 0 ••• f L f Then another equivalent quadratic form of the distance is just: d2(e) = (v e /v e = e '(v 'v )e = e'F e where F is a (K 4-1) x (K + 1) real, symm etric and Toeplitz m atrix ^ do Pl P2 ■■■ Ok N T - l 1 Pi Oo Ox ••• Ok -i F = v v = < ? 2 Ox Oo • • • Ok -2 ^ Or- O k - i O h - z ■■■ Oo J and L-k Ok = ^ 2 fifi~ k 1=0 (5.27) (5.28) (5.29) (5.30) It is easy to see that 6 k = 0 when L < k < K . Again. F is positive definite because of its definition in (5.28). Some definitions are given here for future use. Denote the reverse vector of x as x b. i.e.. x 6 =f (x n x n_t Xi)'. then x is called sym m etric if x = x 6 and antisymmetric if x = — x 6: otherwise it is nonsymmetric. The following Lemma is useful in the later discussions. L e m m a 9 I f m atrix A is real, sym m etric and Toeplitz, then there exists a set o f eigenvectors constituting an orthonormal basis and each eigenvector is either svm m ctric or antisvmmetric. 97 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Proof. Define an "reverse-operation" m atrix (having the same dimension as A) as B = ( 0 0 0 0 0 0 0 0 1 0 1 0 1 0 0 \ 0 1 1 0 ( 5 . 3 1 ) which has the property that for any vector x B x = x 6. ( 5 . 3 2 ) It is easy to check that B B = I. Define a square m atrix C as centrally symmetric (centrally antisymmetric) when B C B = C (B C B = — C ). Because m atrix A is symmetric and Toeplitz. it is centrally symmetric, so B A B = A ( 5 . 3 3 ) For m atrix A its eigenvector x and the corresponding eigenvalue A satisfies A x = Ax. Since A is real and symmetric, its eigenvectors constitute an orthonormal basis for the underlying linear space. Then A x = A B x = B A B B x = B A x = BAx = ABx = Ax6 ( 5 . 3 4 ) Both x and x 6 are eigenvectors of A associated with the same eigenvalue A . If x is symmetric or antisymmetric, the conclusion is trivial. If x is nonsymmetric. then x and x 6 are orthogonal to each other. They can be replaced by the symmetric vector (x -f x 6 ) / \ / 2 and the antisymmetric vector (x — x 6)/\/2 . Obviously, these two vectors are still eigenvectors of A associated with the eigenvalue A. The resulting eigenvector set is still an orthonormal basis. ■ 98 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5.4.2 T he “W orst” ISI C hannel Given the length of memory L . the “w orst” ISI channel is the one which gives the minimum dm in in all possible (L -f l)-ta p ISI channels. The term "worst" is care fully used since dm\n is not the only factor influencing the performance degradation caused by the corresponding ISI [ChChThAn99]. An ISI channel is called sym m etric (antisymmetric) if the vector f is sym m etric (antisym m etric). T h e o re m 10 Given L. there exists a sym m etric or antisym m etric "worst" ISI chan nel. Proof. By the definition of dm in [Fo72] Given L < oc. it is a simple m atter [AnFo75] to prove th at to find dm tn one only need to check finite many non-zero error sequences. Moreover, this finite set can be shrunk to a much smaller sufficient set S [AnFo75]. Pick an error sequence e from £. the square distance about it associated with an arbitrary' (L + I)-tap channel f (non-zero) is Since the m atrix E is positive definite, all its eigenvalues are real and positive. Denote the “worst" channel given e as f(e). then it is easy to show that f(e) is ju st the eigenvector of E associated with its minimum eigenvalue /i(e) > 0. Check all the error sequence in the sufficient set. the “worst" ISI channel is the eigenvector associated with m illed p(e). To prove the theorem , we only need to show th at f(e) can be symmetric or antisym m etric. This can be directly concluded from Lemma 9 since E is real, symmetric and Toeplitz. ■ It should be noted that in [Mi99] the author claims that the “worst" (L -F I)-tap ISI channel corresponds to a cosine function uniformly sampled between ± £ dmin = min e 'F e = rriin f 'E f min (5.35) d2(e) = f 'E f > 0 (5.36) (5.37) 99 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. where C is a scaling constant. This result is consistent with all the "worst" channels with L < 5 provided in [Pr95. Pg. 601]. We have not been able to construct a proof for general cases, but we conjecture th at this form is not true in general. 5.4.3 O n th e M inim um Error Sequence Given an ISI channel f, the error sequence corresponding to dmin is called the m ini mum error sequence (MES). An error sequence set is called sufficient if it includes the MES. In [AnFo75, YaFo91], several rules have successfully made the sufficient error sequence set S as small as possible. In [AnFo75], it is stated that if a nonsymmetric e satisfies (leo| |ei| ••• |e a t [) = (I^a 'I leA'-i| ••• |eo|) (5.38) then it cannot be the MES since a symmetric (antisymmetric) error sequence has a smaller distance. Here, we strengthen this conclusion a step further and get the following theorem . T h e o re m 11 Given the input alphabet {0.1. • • •. M — 1} and a nonsym m etric error sequence e. I f ek + ca'-a.- is even for 0 < k < K , then there exists one sym m etric and one antisym m etric error sequence, which have smaller distances. Proof: Given ek + eK_k is even then ek — e ^ -k = (e* + e ^-k ) — 2ea'-a- must be even as well. Since ]ejt| < M — 1. we must have ek~1f'~k anci ek~e ^K- k as valid error symbols. Thus both and are valid error sequences. Given the ISI channel f. the square distance for e is < f2(e) = e'Fe. Since F is real, symmetric and Toeplitz. by Lemma 9. its I\ + 1 eigenvectors constitute an orthonorm al basis {x0. Xo. • • •. Xa }- and each of them is symmetric or antisymmetric. Moreover. F is positive definition, so all its eigenvalues are real and positive, i.e., Xk > 0 for all k. By the definition of a basis, we can always find a unique set of coefficients ck such that A' e = ^ 2 = X c (5.39) fc=0 100 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. where X = (x0 Xi ••• x*.-) and c = (c0 ••• cK)'. Note that ||e ||2 = J2k=0 °% - Consequently. we get k = 0 d2(e) = e'Fe = c'X'FXc = c'Ac = ^ A kc2 where A = diag(A0r Al? • • •. A*-). Since A' eb = Be = B y ckx k = ck Bx* = c*x£ k = 0 k = 0 Then we m ust have e 4- e& ^— ' — 5— = 2 - ( 5 . 4 0 ) K ( 5 . 4 1 ) k = 0 k:x k is sy m m e tric e — e° E C f c X f c ( 5 . 4 2 ) ( 5 . 4 3 ) k:xic is a n tis y m m e tric and consequently rf2(e_-r_e_) _ ^ Xkcl < d2(e) k:Xic is s y m m e tric d2(e 9 e ) = 2 1 XkCk - k:Xk is a n tis y m m e tr ic ( 5 . 4 4 ) ( 5 . 4 5 ) To find c/mjn. one does not need to consider the nonsymmetric error sequences e which satisfies the condition that ek + e ^ -k is even for 0 < k < K . ■ Unfortunately, the above argum ents cannot be extended directly to arbitrary nonsymmetric error sequences since the space consisting of all error sequences is not a linear space. Although some nonsymmetric error sequences have been found for £ in [AnFo75] and even some nonsymmetric MESs have been found in [\aFo91]. it does not exclude the existence of a symmetric or antisym m etric MES since MES is not necessarily unique (module ± e and ± e 6). So. we still conjecture that: Conjecture 2 For each IS I channel, one can find a sym m etric or antisymmetric MES. 101 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5.4.4 A b ou t 2D ISI C hannels By treating each column or row in a d ata page (lim ited size) as a single symbol, we can apply Theorem 10 to the 2D ISI channels. Then the "worst” 2D ISI channel (in the sense of d 'min. Section 3.2.3.1) should be both column-wise and row-wise symmetric or antisym m etric. Due to the lack of a natural order in the 2D index, the "worst'' 2D ISI should be invariant under tt/2. ~ and 3tt/2 rotations. Combining these two properties, we conclude that T h e o re m 12 The "worst" L x L 2D ISI channel m ust be centrally sym m etric or antisymmetric. This predicts th at the "worst” 2D ISI found in Section 3.2.3.1 is not only the "worst" 3 x 3 centrally symmetric 2D ISI channel (the channel defined in (3.12)). but also the "worst” 3 x 3 2D ISI channel. It is easy to figure out th at another "worst" 3 x 3 2D ISI channel is ( 1/2 - l / v /2 1/2 ^ - l / v /2 1 - l / v /2 ^ 1/2 - l / y / 2 1/2 ) (5.46) which is also centrally symmetric and the corresponding minimum error page (which gb'es < n J is / 1 1 \ i i / (5 J7) We also conjecture th at C o n je c tu re 3 The “ worst" L x.L 2D IS I channel is the outer product of the “ worst" L-tap ID IS I channel. 5.5 On R egular Sparse ISI C hannels 5.5.1 T he Sim ple Sparse ISI C hannels T h e o re m 13 Given a simple S-ISI channel: { (i,id g, f(i))\d g > 1: 0 < i < L s}. It has the same distance spectrum as ( f( 0) / ( l ) • • • f ( L s))', a dense ISI channel. 102 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Proof. For sim plicity of presentation, the simple S-ISI channel {(0. 0. / 0), (0 .2 ./i)} is used for the proof. It is straightforward to extend the following ar guments to arbitrary simple S-ISI channels. W e call this S-ISI channel Channel A. and the ISI channel (fo / 1)' Channel B. Then the m atrix defined in (5.29) for Channel A and Channel B must have the form as: F.4 / do 0 Oi 0 • 0 Oo 0 Pi 0 1 0 Oo 0 0 0 1 0 O o V : • F b = ( O o Pi 0 ... \ Ol Po Pi (5 0 Pi Po d e f where o0 = fo + f { and 0\ = /o /i- Given a error sequence e. we define ee = {e0 e - > • • ■ } and eG =f (e L e3 • • •}. Note that ee and eG are independent. Then the squared distance of e associated with Channel A is d“ i(e) = e'F.-ie = e„F b& o + e^F a e e = d2 B(e0) + d 2 B(ee) (5.49) This equality can be interpreted graphically via Fig.5.4. Channel A can be viewed as consisting of two Channel B's running independently in ping-pong fashion tem porally. From (5.49). first it is easy to see th at Channel A and Channel B have the same (L This is because min d2 4(e) = min (d2 B(e0) + d2 B(ec)) e = t O e s O = min ,(d% (ea) + d2 g (ee)) e Q o r e e 9=0 = min dB(e0) = m in d ^(ee) e„ 5 ^ 0 e, #0 (5.50) This is equivalent to constructing a sequence to make one and only one branch in Fig.5.4(b) to have non-zero inputs. So the MES of Channel A m ust be of the form {e0 0 c2 0 ej • • •}. where (eo eo e4 • • •} is the MES of Channel B. Given the MES 103 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Q q. a-i. a?, a3. (a) D v fo D D v fi V •Z*q. X 1 . X o . X 3 . «0.a2. " ^ d » D V V fo fl V X q . X 2 . ■ ■ ■ 4 * ■ *> a0. a l . a2. a3. • — x0. 2*1. x 2. x 3. - • • < > (b) fo /. A A > D » ■ D ai. a3. ■ ■ ■ Figure 5.4: An equivalent model for a simple S-ISI channel. for Channel B (A), one can readily write down the MES for Channel A (B). On the other hand, every possible value in the distance spectrum of Channel A can be found in th at of Channel B. and vice versa. Denote e(d) to be set of error sequences which have the same distance d for the given channel. By sim ilar arguments, it is easy to verify that for the same d. there exists a one-to-one m apping between e^id) and es(d ). In other words, we say that Channel A and Channel B have the same distance spectrum . ■ 5.5.2 T h e R egular Sparse ISI C hannels Although Theorem 13 does not hold for a regular S-ISI channel with lg > 1. its nice structure is still helpful in the following approximate analysis, which is followed by the performance analysis for the 2D ISI/AW GN channels [Ch96, ChChThAn99]. Imagine the taps of a regular S-ISI channel as the spots on a ribbon. Illustrated in Fig.5.5. we can wrap this ribbon around an imaginary cylinder. Since these spots 104 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. fo f 1 f i f 10 f l \ f 11 /-40 / l l /-1 2 wrap around /2 /l /o /2 2 / a i /2 0 /1 2 / 41 /-10 Figure 5.5: A two dimensional ISI model for a regular S-ISI channel. are positioned regularly, they can be aligned to form a m atrix on the cylinder sur face. As a special case, wrapping a simple S-ISI around simply results in a ID ISI along the cylinder axis. Although the radius of the cylinder is limited by the finite dg. the cylinder surface can be well approximated by a plane when dg 3 > 1. The difference between a plane and a cylinder surface can be well analogous to the differ ence between a convolutional-coded sequence and a tailbiting convolutional-coded [CoSu94. CaFoVa99] circle. When the circle is large enough, the side information that the head and tail of the coded sequence are sewn together becomes negligible for the symbols close to the center of the data sequence. Recently, in [SmGe99] Smyth and Ge are able to construct a proof to show th at the effect of a variable tends to fall off exponentially with distance on the path of belief propagation. From another viewpoint, the rule-of-thumb used in determining the decision delay of a YA [HeJaTl] tells that the correlation can be ignored when D > 5L ~ 7L. Consequently, when a regular S-ISI channel satisfies the condition th at dg > 5lg ~ 7lg {e.g.. , the one shown in Fig.5.5). such approximation is reasonable. It is notable th at for a simple S-ISI channel, its 2D ISI model degrades to a dense ID ISI channel. Therefore the columns (on the cylinder surface) remain uncoupled by this ID ISI along the row direction. Thus the approximation taken place when replacing a cylinder surface by a plane does not apply in this case. As expected by Theorem 13, the standard 105 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. \A (i.e., PTYA in [McKeHo98]) or the SISO can be applied on the equivalent ID dense ISI channel to yield the optim al solution. When we approxim ate a regular S-ISI channel by its corresponding 2D ISI model, we omit the inform ation that the (finite) "plane" is sewn together at its two opposite edges. In the optim al detection, the more inform ation (e.g., observations, side information) we can obtain given the information is correct, the better we can estim ate. Therefore, the M LPE for the 2D ISI model should be no better than the MLSE for the original ID ISI. W hen dg 1. we may apply the results obtained from the 2D ISI model to the original regular S-ISI channel. It is easy to see th at the above discussions about the regular S-ISI is also appli cable to alm ost-regular 2D ISI channels. For example, a irregular S-ISI channel with non-zero taps at positions {0.1,2.20,21.40.42} can be treated just like the channel in Fig.5.5 by setting f 2 2 = / u = 0. Also an irregular S-ISI channel with non-zero taps at positions {0.1.2,19.20.21,40,41,42} should have almost the same performance as the regular one in Fig.5.5. 5.6 Likelihood C om bining 5.6.1 A Toy P roblem o f Join t D etection Give an input binary symbol 6. put it into two binary sequences bi and bo arbitrarily as in Fig.5.6. Assume that this symbol b and all other symbols in these two sequences '» l> ! bo Figure 5.6: Two independent d ata sequences with one common symbol, are independent. Passing these two sequences through an arbitrary channel suffering 106 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. independent noises, two observation sequences Zi and z2 are obtained. Obviously the optimal estim ate about 6 is obtained by finding the joint probability ratio /« . ^ P r(6 = 1. z 1 ? z2) 7 (6. Zi. z2) = pTTT-----n , v (°-01) r r (0 = U. Zi, z2j By the given conditions, we have P r(6 = B m. z L .z 2) = E E P r (6 = B m. b 1 ? b2, z 1 : z2) bi b o = ^ 2 ^ 2 P r (Zl> z' 2\b = b i- b s) P r(6 = B m. b t. b>) bt b2 P r(6 = B m.z.i) P r(6 = B m. z2) P r (6 = B m) Consequently, we obtain the joint probability ratio by (5.52) ,l . /(M i 7 (6. z2 _ox 7(6. z 1?z2) = ----------— -------- (o.o3) 7 (6) where 7 (6. 6 ) =r P r(6 = 1 .6 )/P r(6 = 0, o) for arbitrary condition 6 . It turns out that the optim al joint probability ratio based on two observation sequences is the product of two probability ratios, each of which is based on only one observation sequence. The sim ilar conclusion can be drawn in the MLSE detection (metric- based). where the joint generalized likelihood (GL) ratio can be obtained via: T,/, x d e f minbl m inb ;: M (6 = 1. b i. b 2. z x. z2) r(6. Zi-Z2) — . . n 1 . \ minbl nunb, M (6 = 0. b lr b 2. z2) = r(6.Zl) + r( 6,Zl) - r(6) (5.54) By removing the condition that except for 6 sequences b t and b 2 are independent, we fall into the situation of a Turbo code [BeGlTh93] which has two constituent codes. Although b i and b 2 are correlated since one is just a perm utation of another, in Turbo codes random-like interleavers help to scramble the input sequence and weaken the (regular) correlation between the input and output sequences as much as possible. Consequently, the ideal soft inform ation combining rule in (5.53) or (5.54) has been applied approximately for the likelihood combining in Turbo codes. 107 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. The same rules have also been used in the construction of the com posite 2D SISO algorithm in Section 3.2.2 and of the multi-SISO algorithm in Section 4.3.2.3. Rearrange the two sequence in the previous toy problem into a cross shape shown in Fig.5.7 in which b is put at the cross point. Assume this d ata cross passes a 2D bi b 2 ISI Figure 5.7: D ata on a cross. ISI channel illustrated in Fig.5.7 and we only observe when the center of the 2D ISI is aligned on the cross. Consequently we obtain two observation sequences Z[ and z2 where one of their pixels are the same. Again the optim al detector needs to compute 7 (b. z 1 ? z2). However, because the effect of the 2D ISI, these two observation sequences are correlated. Then two interesting problems naturally come out: 1. Is there an efficient way to obtain 7 (6. Z i,z2)? 2. Given 7 (6,Z i) and 7 (6. z L ). what is the best way to glue them together? The first problem is just a specific case of the more general 2D least metric problem discussed in Section 5.1. So there is no efficient way to obtain 7 (6, z i,z 2). However, an APP-SISO can efficiently find 7 (6. z t) and 7 (6, Zi) (fix the soft information about the coupled symbols in the other branch). Is there a better scheme to glue them together than by multiplying them? The following toy problem may give some insight to this question. 108 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. 5.6.2 L ikelihood C om bining Toy P rob lem Assume the received signal is ) (5.55) ~y / \ y / \ n y When the input 6 = 0, (x0 y0)' is transm itted: otherwise, (xi y\)' is. The additive noise is jointly Gaussian with the correlation m atrix as: r(0. 0) = r(l, 1) = 1 and r(0. 1) = r(1.0) = p. We are given the likelihood ratio 7 (6. zx ) and 7 /(6. zy). Can we obtain the optim al likelihood ratio 7 (6. zx. zyY ? If yes. how? If not. w hat’s the best combining scheme? Actually, since 7 (6. zx ) = e x p (- — -----------9 ~ exp((x! - x Q )zx). (5.5G) if x0 ^ x x. the value of zx can be recovered perfectly once 7 (6, zx) is given. Therefore if x0 # x x and y0 # £/i. the observation (zx, zy) can be recovered and 7 (6. cx, zy) can be obtained. However, when one of coordinates of these two signal points coincides. e.g.. x0 = -Ti, no knowledge about zx cannot be obtained except that zx is Gaussian distributed with mean xo and variance 1. Given the recovered zy, the best solution should be an average likelihood ratio: ^ _ / x Pr(& = l . z x = x. zy) f Z x {x)dx f x /a W Jx P r(6 = 0. cx = x. zy) f Zx{x)dx (d’° ' j By changing the design of the signaling or rotating the observation coordinate, we can always have x0 7 ^ x\ and yo # Z /i- and thus obtain the optimal likelihood from two localized likelihoods. However in practice the situation is much more complicated than this toy prob lem. We are interested in finding a simple and generic way to accomplish the likelihood combining. To restate the toy problem in the metric domain, we have 109 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Mx(6 = 0) = (~x — x0)2 and My(6 = 0) = (zy — * /0)2- It is easy to check th at the optim al combining rule is M (6 = 0) = Mx(6 = 0) + My(6 = 0) - 2py/.Mx(& = 0)My(6 = 0) (5.58) We may treat this combining rule as a generic one and apply it to m ore complicated scenarios. For example, in the composite 2D SISO. the two m etrics from the two SISO submodules. Mi 1(a(/.J)) and M?(a(z._/)) must be correlated. If we can find their correlation coefficient theoretically or numerically, by only considering to the second-order statistics, may the more sophisticated combining rule in (5.58) provide a better M,(a(i, j)) than (5.54)? 110 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. A p p en d ix A M atrix-based C om putation o f th e F L -A P P -SISO A lgorithm In the FL-APP-SISO. a D -step backward recursion is needed to calculate at each time slot. It is possible to accomplish this D-step recursion in just one step via m atrix-based com putation. W ith the m atrix notation B jS ? = [.J& ?(0) 1) • • • - 1)]' (A .l) I \ =f where j) = 0 if 5, -> S j is not valid (A.2) it is simple to rewrite the backward recursion (2.15) as B jt+f = r * +1B £ ? (A.3) D D = n r ^ Bh L , = n r ^ (a -j ) n = l n = l where B£Xd* i = 1 ‘ ' l]* according to (2.16). Then at the next time slot. B £ i2 DTl = n r *+i« . n= 1 = rf c + 0 + 1 (A.5) This operation can be viewed as a one-step forward recursion. Although I’j.j. j is a |«S| x |«S| m atrix, it is a sparse m atrix with a fairly regular structure. It can be shown that its inverse can be accomplished by inverting j-jj- m atrices of dimension j.4| x \A\. When the input symbol is binary (|.4| = 2). this m atrix-based com putational structure could be feasible. I l l Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. A ppendix B C om plexity R eduction Techniques for SISO A lgorithm s In this section, only the APP version of a SISO is used for presentation while the same techniques are applicable to the corresponding MSM-SISO algorithms. In the FI-APP. after finishing the forward and backward recursions, the soft information is computed via: P r(af c = .4m,z*'+ i) = 'ST Pv{sk+l = S u z * +L) (B.l) leCo(m ) Since sk+i = a£_L_n, we can get the soft information P r(af c _rf.z ^ L) for d = 0. — 1 from Pr(sfc+1. z f ~L) b\- summing sk+i over Cd(m). Consequently, in the forward recursion, we only need to store a * (i) for k = L, 2L, ■ ■ -. namely, the memory requirement is reduced by L times. Moreover, since Pr(sjt+ 1 = 5 ,.z ^ ~ L) = the com putational load in the completion step is cut by L times as well. It is notable that this saving is achieved without changing the soft output. Applying this approach to the FL-APP, a similar saving can be achieved. Actu ally. it results in a variable-lag (YL) A PP algorithm that performs better than the FL-APP. In the FL-APP. the desired soft information is computed through: Pr(a* = .4m,z f+° ) = J 2 Pr(s*+i = S i;z*+°) (B.2) i£Co(m ) Similarly. P r(sfc 4 .1. z^+D) can be used to calculate P r(af c _d. z ^ 0 ) for d = 0.1. • • •. L — 1 with a variable delay of D + L — 1. D -I- L — 2, • • •. D. respectively. Moreover, in the FL-APP it is the D -step backward recursion, which is needed for every time slot, that dominates the com putation. However in the YL-APP. the backward recursion is fired only once every L time slots. In comparison, the YL-APP saves the computation approximately L times. On the other hand, it has been shown that the system performance is almost independent on the detection delay D when D > 5 ~ 7£ [HeJaTl, LiYuSa95, AnCh98]. Therefore, the YL-APP can provide the same reliable soft output as the FL-APP and the FI-A PP when D > 5 ~ 7L. 112 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. A p p en d ix C The E quivalence o f th e F L -A P P and L2 V S -A P P T h e o re m 14 The FL-APP and the L2 V S-APP are equivalent when used for an isolated FSM. Proof: For an isolated FSM. we are only interested in the soft information about the input of the FSM. i.e., a*. Although the FL-A PP and L2VS-APP have different architectures, the}- do yield the exactly same soft information P r(afc . z \ +D). This directly implies the equivalence of the FL-A PP and L2VS-APP. Actually, the L2YS- APP can be also derived from the FL-APP rigorously. The main idea is to change the serial backward recursions in the FL-A PP into the parallel forward recursions in the L2VS-APP. In order to obtain the soft information about ak at time k + D. 3k kZ?{l) is required. Starting from the definition of 3. we can get = P r ( * S f |s * +, = Si) - ^ Prf't-D— i = s r z;::P|s^ i = Si) (c.i) J Each term in the sum m ation can be computed forward-recursively as: Pr(s*+„ = Sj, = Si) = ^ 2 P r(sfc+„_! = S i,s k+n = = Si) ien j) = 5 1 P r(sfc+n_i = S i^ z ^ i^ ls k + i = (C.2) i€FU) This formula is valid for L+ I<n <£ > + 1 while sk~n-i and 1 are independent. When n = L + 1. the initial term of the recursion in (C.2) can be calculated directly via: P^Sfc+i+^z^flst+O = Pr(a£;f.z£;f|a£_L+l) = Pr(ak+uzk+i\sk+1) P r ( a ^ . z ^ |a ^ i [ +2) 113 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. = P r(afc+„. Sit+n|sfc+n) (C.3) n = l Consequently, instead of a backward recursion in the FL-APP. .^jt+PCO can °b~ tained through a forward recursion which starts at time k+ 1 and ends at time k + D. Actually, by keeping track of a set of such forward recursions. 3's can be computed by a parallel forward-recursive structure. However, the above forward structure is complicated because recursion (C.2) is conditioned on two states, resulting a com plexity factor of | +0(J') Pr(Qt = A ^ \s k^ i = Sj.**+D) (C.5) j The first term can be calculated forward-recursively as in the FL-APP. and the second term is closely related to (C.4). It is easy to see th at the derivation of (C.4) is independent on the start point of the sequence z. Therefore, we can readily obtain (C.6) Pr(a* = -4rn|.S fc +n = S j.z * +n_1) Yhef(j) PV(ak = = 5i.zf+n"2)Q ^+,,"2(07fc+n-l(-L j) “ * k r n- l u ) for L + 1 < n < D + 1. Similarly, this recursion is initialized at time k via: p r (n _ 4 I C „ k + L \ (0 !k-rL (L j ) / p r r(.Qfc — — jj, Z[ J — k-t-L / -\ ('“ •C Qi 0) 114 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. where sk+L = S, is uniquely determined by the value of ak and Sk+L-ri- (C.5)-(C.7) is exactly the L2 VS-APP. As the cost of having a parallel com putational architecture, the L-YS-APP is |.4| tim es more complex than the FL-APP. ■ It should be noted th at the L2YS-APP is unable to provide the soft output about x k directly as the FL-APP. This is because the com putational architecture of the L2YS-APP is intentionally designed to calculate Pr(a^. z \~ D) efficiently without considering the possible need for PQ (xt). In [LiYuSa95] the L2YS-APP was at the first place designed for the detection in a serially concatenated system in which no P0(xjt) is needed. However, when applied to applications which needs the soft output about x k, the L2YS-APP needs necessary modifications. One such possible modification is to obtain the soft output about x k via [ChCh99]: P o t r * ) " J 2 II (C.8) a k k_ L:xk =x(ak k _ L ) i= k - L 115 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. R eference List [AbFr< 0] [AnCh98j [AnFo75] [BeBeMa98] [BaCo.JeRa74] [BoDiMoPo98] [BeGe96] [BcG196] K. Abend and B. D. Fritchman. "Statistical detection for com munication channels with intersymbol interference." Proc. IEEE, vol. 58. no. 5. pp. 779-785. May 1970. A. Anastasopoulos and K. M. Chugg, "Iterative equaliza tion/decoding of TCM for frequency-selective fading channels." in Proceeding of the 32th Asilomar Conf. on Signal, System s and Comp., (Los Alamitos. CA). IEEE Com puter Society Press. Nov. 1998. R. R. Anderson and G. J. Foschini. "The minimum distance for MLSE digital data systems of limited complexity." IEEE Trans. Inform. Theory, vol. IT-21, no. 5. pp. 544-551. Sept. 1975. X. Benvenuto. L. Bettella. and R. Marchesani. "Performance of the Viterbi algorithm for interleaved convolutional codes." IEE E Trans. Vehic. Technol., vol. 47. no. 3. pp. 919-923. Aug. 1998. L. R. Bahl. J. Cocke, F. Jelinek. and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate." IEEE Trans. Inform. Theory, vol. IT-20, no. 3, pp. 284-287. Mar. 1974. S. Benedetto. D. Divsalar. G. Montorsi. and F. Pollara. "Serial con catenation of interleaved codes: performance analysis, design, and iterative decoding.'" IE E E Trans. Inform. Theory, vol. 44. no. 3. pp. 909-926, May 1998. A. Becker and D. Geiger. "Optimization of Pearl's method of con ditioning and greedy-like approximation algorithm s for the vertex feedback set problem." Artif. Intell.. vol. 83, no. 1. pp. 167-188. May 1996. C. Berrou and A. Glavieux, "Xear optimum error correcting cod ing and decoding: Turbo-codes." IEEE Trans. Cormnun., vol. 44, no. 10, pp. 1261-1271, Oct. 1996. 116 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [BeGlTh93] [BeLuMa93] [BeMa96] [BeMo96] [BoMoDiPo96] [BeMoDiPo98] [BePa79] [BeSa94] [CaFoVa99] [Ch96] [Ch98] C. Berrou, A. Glavieux, and P. T hitim ajshim a, "Near Shannon limit error-correcting coding and decoding: Turbo-codes (1)." in Proc. IE E E ICC'89, (Geneva. Switzerland), pp. 1064-1070. Mav 1993. X. Benvenuto, G. Lubello. and R. Marchesani. "M ultitrellis de composition of the Viterbi algorithm for m ultipath channels,- ' in Proc. ICC'93. (Geneva, Switzerland), pp. 746-750. May 1993. X. Benvenuto and R. M archesani, "The V iterbi algorithm for sparse channels," IEEE Trans. Com m un., vol. 44, no. 3. pp. 287- 289. Mar. 1996. S. Benedetto and G. Montorsi, "Unveiling turbo codes: Some re sults on parallel concatenated coding schemes.” IE E E Trans. In form . Theory, vol. 42, no. 2. pp. 408-428. Mar. 1996. S. Benedetto, G. Montorsi. D. Divsalar. and F. Pollara. “A soft- input soft-output maximum a posteriori (MAP) module to decode parallel and serial concatenated codes," tech. rep.. TDA Progress Report 42-127, Xov. 1996. S. Benedetto, G. Montorsi, D. Divsalar. and F. Pollara. "Soft-input soft-output modules for the construction and distributed iterative decoding of code networks," European Trans. Telecommun.. vol. 9, no. 2. pp. 155-172. M ar/A pr 1998. C. A. Belfiore and J. H. Park, Jr., "Decision feedback equalization," Proc. IEEE, vol. 67. no. 8. pp. 1143-1156. Aug. 1979. X. Benvenuto and A. Salloum. “Performance of the mutlitrellics Viterbi algorithm for sparse channels." in Proc. GLOBECOM '93. (San Francisco, CA), pp. 1493-1497, Xov. 1994. A. R. Calderband, G. D. Forney, Jr.. and A. Vardy, "Minimal tail- biting trellises: the Golay code and more," IE E E Trans. Inform. Theory, vol. 45, no. 5, pp. 1435-1455. July 1999. K. M. Chugg, "Performance of optim al digital page detection in a two-dimensional ISI/AW GX channel." in Presented at the 30th Asilomar Conf. on Signal, System s and Comp.. Xov. 1996. K. M. Chugg, "The condition for the applicability of the Viterbi algorithm with implications for fading channel mlsd." IE E E Trans. Commun.. vol. 46, no. 9. pp. 1112-1116, Sept. 1998. 117 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [ChHa66] [ChCh98] [ChCh98b] [ChCh99] [ChCh99b] [ChCh99c] [ChChXe97] [ChChXe98] [ChChXe99] [ChChOrCh98] [ChChThAn99] R. W. C hang and J. C. Hancock, “On receiver structures for chan nels having memory." IE E E Trans. Inform. Theory, vol. IT-12. no. 4, pp. 463-468. Oct. 1966. X. Chen and K. M. Chugg, “Xear-optim al d ata detection for two- dimensional ISI/AWGX channels using concatenated modeling and iterative algorithms." in Proc. IE E E ICC'98. Vol. 2. (A tlanta, GA). pp. 952-956, June 1998. K. M. Chugg and X. Chen. ‘ 'Efficient architectures for soft output algorithm s." in Proc. IE E E ICC'98. Vol. 1, (Atlanta, GA). pp. 126- 130. June 1998. K. M. Chugg and X. Chen, “On a-posteriori probability (APP) and m inimum sequence metric (MSM) algorithms," subm itted to IE E E Trans. Commun., Mar. 1999. X. Chen and K. M. Chugg, “Reduced-state soft-input/soft-output algorithm s for complexity reduction in iterative and non-iterative data detection/' subm itted to IE E E IC C r 2000, Sept. 1999. X. Chen and K. M. Chugg, “Iterative soft-input/soft-output algo rithm s for arbitrary sparse channels," subm itted to IE E E Trans. Commun., Xov. 1999. K. M. Chugg, X. Chen, and M. A. Xeifeld. “Two-dimensional linear MMSE equalization for page-oriented optical memories,'7 in Pre sented at the 31th Asilomar Conf. on Signal, System s and Comp.. Xov. 1997. X. Chen, K. M. Chugg. and M. A. Xeifeld, "Xear-optim al paral lel distributed data detection for page-oriented optical memories," IEE E J. Select. Topics Quantum Electron., vol. 4, no. 5, pp. 866- 879, S ept./O ct. 1998. K. M. Chugg, X. Chen, and M. A. Xeifeld. “Two-dimensional equalization in coherent and incoherent page-oriented optical mem ory." ,/. Opt. Soc. Amer. A, vol. 16, no. 3, pp. 549-562, Mar. 1999. K. M. Chugg, X. Chen, A. Ortega, and C.-YV. Chang, “An iterative algorithm for two-dimensional digital least metric problems with applications to digital image compression,7 ' in Proc. IE E E IC IP ’ 98, Vol. 2, (Chicago, Illinois), pp. 722-726, Oct. 1998. K. M. Chugg, X. Chen, P. Thiennviboon, and A. Anastasopoulos, “Two-dimensional data detection: Achievable performance and 118 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [Co90] [CoLeRi90] [CoSu94] [CoTh91] [DaLu93] [DeLaRu77] [DuHeS9] [EyQuSSj [FoGcFi98] [FiGc96] [FoCC] near-optim al algorithms.” to be subm itted to IEEE Trans. Inform. Theory, 1999. G. F. Cooper, "The com putational complexity of probabilistic in ference using Bayesian belief networks,” Artif. In teli. vol. 42. no. 2-3. pp. 393-405, Mar. 1990. T. H. Cormen, C. E. Leiserson. and R. L. Rivest, Introduction to Algorithms. Cambridge. MA: The MIT Press, 1990. R. C. Cox and C.-E. W. Sundberg, "An efficient adaptive circular Viterbi algorithm for decoding generalized tailbiting convolutional codes.” IEEE Trans. Vehic. Technol.. vol. 43. no. 1. pp. 57-68. Feb. 1994. T. M. Cover and J. A. Thomas, Elements of Information Theory. Xew York: John Wiley & Sons. Inc.. 1991. P. Dagum and M. Luby, "Approximating probabilistic inference in Bayesian belief networks is X P-hard.” Artif. IntelI., vol. 60. no. 1. pp. 141-153, Mar. 1993. A. P. Dempster, X. M. Laird, and D. B. Rubin, "Maximum like lihood from incomplete data via the EM algorithm (with discus sion).” J. Roy. Statist. Soc., vol. 39. no. 1. no. 1. pp. 1-3S. 1977. A. Duel-Hallen and C. Heegard. "Delayed decision-feedback se quence estim ation,” IE E E Trans. Commun.. vol. 37. no. 5. pp. 428-436, May 1989. M. V. Eybuoglu and S. U. H. Qureshi. "Reduced-state sequence es tim ation with set partitioning and decision feedback." IE E E Trans. Commun.. vol. 36. no. 1. pp. 13-20. Jan. 1988. I. J. Fevrier, S. B. Gelfand, and M. P. Fitz, "Fast com putation of efficient decision feedback equalizers for high speed wireless com m unications,” in Proc. IE E E ICASSP'98. Vol. 6, (Seattle. WA). pp. 3493-3496, May 1998. M. P. Fitz and S. B. Gelfand, "Reduced complexity symbol-by- symbol soft output algorithm s.” in Proc. IEEE Globecom’ 96. (Lon don, British), pp. 21-25, Xov. 1996. G. D. Forney, Jr., Concatenated Codes. Cambridge. Massachusetts: M.I.T. Press, 1966. 119 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [Fo72] [Fo72h] [Fo73] [Fo75] [FrKs98] [Ga63] [GaJo79] [GeGe84] [G066] [G068] [GrSl] [Gr99] G. D. Forney. Jr., "Lower bounds on error probability in the pres ence of large intersymbol interference,” IEE E Trans. Commun., vol. COM-20, no. 1. pp. 76-77, Feb. 1972. G. D. Forney, Jr.. "Maximum-likelihood sequence estim ation of digital sequences in the presence of intersymbol interference.” IE E E Trans. Inform. Theory, vol. IT-18. no. 3, pp. 363-378. May 1972. G. D. Forney, Jr.. "The Viterbi algorithm ,” Proc. IEEE, vol. 61. no. 3, pp. 268-278. Mar. 1973. G. J. Foschini. "Performance bound for maximum-likelihood re ception of digital d ata.” IE E E Trans. Inform. Theory, vol. IT-21, no. 1. pp. 47-50. Jan. 1975. B. Frey and F. Kschischang, "Early detection and trellis spilic- ing: Reduced complexity iterative decoding.” IE E E J. Select. A r eas Com mun., no. 2, pp. 153-159, Feb. 1998. R. G. Gallager, Low-Density Parity-Check Codes. Cambridge. M assachusetts: M.I.T. Press, 1963. M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness. Xew York: W.H. Freeman and Company. 1979. S. Gem an and D. Geman. "Stochastic relaxation. G ibbs distribu tions. and the Bayesian restoration of images.” IE E E Trans. Patt. Anal. Machine Intell., vol. P A M I-6 , no. 6, pp. 721-741. Xov. 19S4. B. Goldberg, "300 kHz-30 MHz M F /H F ,” IEE E Trans. Commun. Technol., vol. COM-14, no. 6, pp. 767-784. Dec. 1966. J. W. Goodm an. Introduction to Fourier Optics. McGraw-Hill. Inc., 1968. D. J. G ranrath, "The role of human visual models in image pro cessing.” Proc. IEEE, vol. 69. no. 5. pp. 552-561, May 1981. A. J. G rant, "Information geom etry and iterative decoding,'' in Proc. IE E E Communications Theory Workshop '99. (Aptos. CA), May 1999. 120 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [HaCoRi82] [HaOfPa96] [HeGuHe96] [He.JaTl] [Jo69] [I\o71] [KsFr98] [Lc74j [LoHi77] [LiYuSa95] [LuBoBr98] [MaNe96] J. F. Hayes, T. M. Cover, and J. B. Riera. "O ptim al sequence detection and optim al symbol-by-symbol detection: Similar algo rithm s,” IEEE Trans. Commun., vol. COM-30, no. 1, pp. 152-157. Jan. 1982. J. Hagenauer, E. Offer, and L. Papke. "Iterative decoding of bi nary block and convolutional codes,v IE E E Trans. Inform. Theory, vol. 42. no. 2. pp. 429-445. Mar. 1996. J. F. Heanue, K. Giirkan. and L. Hesselink. "Signal detection for page-access optical memories with intersymbol interference.” Appl. Opt., vol. 35. no. 14. pp. 2431-2438, May 1996. J. A. Heller and I. M. Jacobs, "Viterbi decoding for satellite and space com m unication.” IEEE Trans. Com mun., vol. COM-19. no. 5, pp. 835-848. Oct. 1971. F. Jelinek. "A fast sequential decoding algorithm using a stack.” IB M J. Res. and Dev., vol. 13. pp. 675-685. Nov. 1969. H. Kobayashi. "A survey of coding schemes for transmission or recording of digital d a ta .” IEEE Trans. Commun. Technol., vol. COM-19, no. 6. pp. 1087-1100. Dec. 1971. F. R. Kschischang and B. J. Frey. "Iterative decoding of compound codes by probability propagation in graphical models.” IEEE J. Select. Areas Commun.. vol. 16, no. 2. pp. 219-230. Feb. 1998. L.-X. Lee, "Real-time m inim al-bit-error probability decoding of convolutional codes.” IE E E Trans. Com m un.. vol. 22. no. 2. pp. 146-151, May 1974. \V. U. Lee and J. F. S. Hill. "A maximum-likelihood sequence es tim ator with decision-feedback equalization/' IE E E Trans. Com mun., vol. com-25. no. 9, pp. 971-979. Sept. 1977. Y. Li, B. Vucetic, and Y. Sato. "Optim um soft-output detection for channels with intersymbol interference.” IE E E Trans. Inform. Theory, vol. 41, no. 3, pp. 704-713, May 1995. R. Lucas, M. Bossert, and M. Breitbach. "On iterative soft-decision decoding of linear binary block codes and product codes.” IEEE J. Select. Areas Commun.. vol. 16, no. 2, pp. 276-296. Feb. 1998. D. J. C. MacKay and R. M. Neal, "Near Shannon limit perfor mance of low density parity check codes.” Electron. Lett., vol. 32, no. 18, pp. 1645-1646, Aug. 1996. 121 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [Mc74] [McKeHo98] [McMaCh98] [Mi99j [Mo97] [Mo98j [MoGu98] [XeChKi96] [PaXe95] [Pe86] [Pe93] [Pr91] [Pr95] P. L. Me Adam, M A P bit decoding o f convolutional codes. PhD thesis, University of Southern California. Los Angeles. California. 1974. X. C. McGinty, R. A. Kennedy, and P. Hoeher. "Parallel trel lis Viterbi algorithm for sparse channels," IEE E Commun. Lett.. vol. 2, no. 5, pp. 143-145. May 1998. R. J. McEliece, D. J. C. MacKay, and J. F. Cheng, "Turbo de coding as an instance of Pearl's "belief propagation" algorithm." IE E E J. Select. Areas Commun.. vol. 16, no. 2, pp. 140-152. Feb. 1998. C. L. Miller. Image restorage using trellis-search methods. PhD thesis, University of Arizona, 1999. M. L. Moher, Cross-Entropy and Iterative Detection. PhD thesis, Carleton University. Ottawa. Canada. May 1997. M. L. Moher, "An iterative multiuser decoder for near-capacity communications." IE E E Trans. Commun.. vol. 46. no. 7. pp. 870- 880, July 1998. M. L. Moher and T. A. Gulliver, "Cross-entropy and iterative de coding," IE E E Trans. Inform. Theory, vol. 44. no. 7. pp. 3097- 3104, Xov. 1998. M. A. Xeifeld, K. M. Chugg, and B. M. King. "Parallel d ata detec tion in page-oriented optical memory," Opt. Lett., vol. 21. no. 18. pp. 1481-1483. Sept. 1996. T. X. Pappas and D. L. Xeuhoff, "Printer models and error diffu sion." IEEE Trans. Imag. Processing, vol. 4, no. 1, pp. 66-80. .Jan. 1995. J. Pearl. "Fusion, propagation, and structuring in belief networks," Artif. Intell., vol. 29. no. 3. pp. 241-288, Sept. 19S6. J. Pearl. "Belief networks revisited." Artif. Intell., vol. 59, no. 1-2, pp. 49-56, Feb. 1993. J. G. Proakis, "Adaptive equalization for TDMA digital mobile radio,” IEEE Trans. Vehic. Techno I., vol. 40, no. 2, no. 2, pp. 333- 341. 1991. J. G. Proakis, Digital Communications. Xew York: McGraw-Hill. Inc.. 3rd ed.. 1995. 122 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [Ra89] [Ri99] [R o\’iHo95] [SmBe97] [SmGe99] [St60] [Ta81] [ThChChOr99] [YaGS] [\'a97] [\ aFo91] [YeS7] L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition." Proc. IEEE, vol. 77. no. 2. pp. 257-286. Feb. 1989. T. Richardson, "The geometry of turbo-decoding dynamics." Pre print. Aug. 1999. P. Robertson. E. Yillebrun. and P. Hoeher, "A comparison of op tim al and sub-optimal MAP decoding algorithms operating in the log domain," in Proc. IEEE IC C ’ 95, (Seattle, \VA), pp. 1009-1013. June 1995. J. E. Smee and X. C. Beauliue. "On the equivalence of the sim ulta neous and separate MMSE optim ization of a DFE FFF and FBF," IE E E Trans. Commun.. vol. 45, no. 2. pp. 156-158, Feb. 1997. P. Smyth and X. Ge, "Distance effects in graphical models." Preprint, to be subm itted to IEEE. Trans. Inform. Theory, Xov. 1999. R. L. Stratono%‘ich, "Conditional Markov processes." Theory Prob. Applications, vol. V, no. 2, pp. 156-178. 1960. R. M. Tanner, "A recursive approach to low complexity codes." IE E E Trans. Inform. Theory, vol. IT-27, no. 5. pp. 533-547. Sept. 1981. P. Thiennviboon. K. M. Chugg, X. Chen, and A. Orgeta. "Iterative algorithm s for two-dimensional digital least metric problems with applications.'' to be subm itted to IE E E Trans. Image Processing. 1999. H. L. Van Trees, Detection, Estimation, and Modulation Theory. Part I. Xew York: John Wiley k Sons, Inc., 1968. A. Yardy, "The intractability of com puting the minimum distance of a code,'' IEEE Trans. Inform. Theory, vol. 43, no. 6. pp. 1757- 1766, Xov. 1997. G. Yannucci and G. J. Foschini, "The minimum distance for dig ital magnetic recording partial responses," IEEE Trans. Inform. Theory, vol. 37, no. 3, pp. 955-960, May 1991. S. Verdu, "Maximum likelihood sequence detection for intersymbol interference channels: A new upper bound on error probability," IE E E Trans. Inform. Theory, vol. IT-33, no. 1, pp. 62-68, Jan. 1987. 123 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. [Ye98j [Yi67] [Wa.JuBeTO] [\Yi96] [\YiLoKo95] [\Yo97] S. Verdu. M ultiuser Detection. Cambridge. United Kingdom: Cambridge University Press, 1998. A. Viterbi, "Error bounds for convolutional codes and an asymp totically optim um decoding algorithm /’ IE E E Trans. Inform. The ory, vol. I T - 13, no. 2. pp. 260-269, Apr. 1967. C. C. W atterson, J. R.Juroshek, and W. D. Bensema, "Experimen tal confirmation of an HF channel model,” IE E E Trans. Commun. Technol., vol. C O M -18, no. 6, pp. 792-803, Dec. 1970. X. Wiberg. Codes and decoding on general graphs. PhD thesis, Linkopings Universitet. Linkopings, Sweden. 1996. X. Wiberg, H.-A. Loeliger. and R. K otter. "Codes and iterative decoding on general graphs,” in Proc. IE E E ISIT '95. p. 468, 1995. P. W. Wong, "Entropy constrained halftoning using multiple tree coding.” IE E E Trans. Imag. Processing, vol. 6. pp. 1567-1579. Xov. 1997. 124 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. INFORMATION TO USERS This manuscript has been reproduced from the microfilm master. UM I films the text directly from the original or copy submitted. Thus, some thesis and dissertation copies are in typewriter face, while others may be from any type of computer printer. The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleedthrough, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send UM I a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note w ill indicate the deletion. Oversize materials (e.g., maps, drawings, charts) are reproduced by sectioning the original, beginning at the upper left-hand comer and continuing from left to right in equal sections with small overlaps. Photographs included in the original manuscript have been reproduced xerographically in this copy. Higher quality 6* x 9* black and white photographic prints are available for any photographs or illustrations appearing in this copy for an additional charge. Contact UM I directly to order. Bell & Howell Information and Learning 300 North Zeeb Road, Ann Arbor. M l 48106-1346 USA 800-521-0600 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. UMI Number 9987600 ___ _ < £ > UMI UMI Microform9987600 Copyright 2001 by Bell & Howell Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. Bell & Howell Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, Ml 48106-1346 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.

Linked assets

University of Southern California Dissertations and Theses

Conceptually similar

PDF

Internet security and quality-of-service provision via machine-learning theory

PDF

Probabilistic analysis of power dissipation in VLSI systems

PDF

Efficient media access and routing in wireless and delay tolerant networks

PDF

Graphical models for iterative data detection

PDF

The design and synthesis of concurrent asynchronous systems.

PDF

Inference of multiple curves and surfaces from sparse data

PDF

Data compression: The theory of algorithm analysis

PDF

Network synchronizations with applications

PDF

A half-tone frequency detector and digital circuit techniques for high-speed clock/data recovery

PDF

Compact VLSI array processors design for multimedia applications

PDF

Stateful computations in functional languages

PDF

Design and analysis of interactive video-on-demand systems

PDF

Semantic heterogeneity resolution in federated databases by meta-data implantation and stepwise evolution

PDF

Multithreshold wavelet codec (MTWC) and perceptual digital watermark casting

PDF

Modulation classification and data detection of signals in unknown ISI environments

PDF

VLSI array sensory information processing for communication and multimedia

PDF

VLSI architectures for video compression applications

PDF

Orthogonal architectures for parallel image processing

PDF

Logic synthesis for low power VLSI designs

PDF

A geometry-centric approach to progressive three-dimensional graphics coding

Asset Metadata

Creator Chen, Xiaopeng (author)

Core Title Iterative data detection: complexity reduction and applications

School Graduate School

Degree Doctor of Philosophy

Degree Program Electrical Engineering

Degree Conferral Date 1999-09

Publisher University of Southern California (original), University of Southern California. Libraries (digital)

Tag computer science,engineering, electronics and electrical,OAI-PMH Harvest

Language English

Contributor Digitized by ProQuest (provenance)

Permanent Link (DOI) https://doi.org/10.25549/usctheses-c17-580345

Unique identifier UC11353850

Identifier 9987600.pdf (filename),usctheses-c17-580345 (legacy record id)

Legacy Identifier 9987600.pdf

Dmrecord 580345

Document Type Dissertation

Rights Chen, Xiaopeng

Type texts

Source University of Southern California (contributing entity), University of Southern California Dissertations and Theses (collection)

Access Conditions The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the au...

Repository Name University of Southern California Digital Library

Repository Location USC Digital Library, University of Southern California, University Park Campus, Los Angeles, California 90089, USA