多媒体技术教程(英文版)

.1.4.5 animation 16
1.4.6 multimedia authoring 17
1.5 further exploration 17
1.6 exercises 18
1.7 references 19
multimedia authoring and tools 20
2.1 multimedia authoring 20
2.1.1 multimedia authoring metaphors 21
2.1.2 multimedia production 23
2.1.3 multimedia presentation 25
2.1.4 automatic authoring 33
2.2 some useful editing and authoring tools 37
2.2.1 adobe premiere 37
2.2.2 macromedia director 40
2.2.3 macromedia flash 46
2.2.4 dreamweaver 51
2.3 vrml 51
2.3.1 overview 51
2.3.2 animation and interactions 54
2.3.3 vrml specifics 54
2.4 further exploration 55
2.5 exercises 56
2.6 references 59
3 graphics and image data representations 60
3.1 graphics/image data types 60
3.1.1 1-bit images 61
3.1.2 8-bit gray-level images 61
3.1.3 image data types 64
3.1.4 24-bit color images 64
3.1.5 8-bit color images 65
3.1.6 color lookup tables (luts) 67
3.2 popular file formats 71
3.2.1 gif 71
3.2.2 jpeg 75
3.2.3 png 76
3.2.4 tiff 77
3.2.5 exif 77
3.2.6 graphics animation files 77
3.2.7 ps andpdf 78
3.2.8 windows wmf 78
3.2.9 windows bmp 78
3.2.10 macintosh paint and pict 78
3.2.11 x windows ppm 79
3.3 further exploration 79
3.4 exercises 79
3.5 references 81
4 color in image and video 82
4.1 color science 82
4.1.1 light and spectra 82
4.1.2 human vision 84
4.1.3 spectral sensitivity of the eye 84
4.1.4 image formation 85
4.1.5 camera systems 86
4.1.6 gamma correction 87
4.1.7 color-matching functions 89
4.1.8 cie chromaticity diagram 91
4.1.9 color monitor specifications 94
4.1.10 out-of-gamut colors 95
4.1.11 white-point correction 96
4.1.12 xyz to rgb transform 97
4.1.13 transform with gamma correction 97
4.1.14 l*a*b* (cielab) color model 98
4.1.15 more color-coordinate schemes 100
4.1.16 munsell color naming system 100
4.2 color models in images 100
4.2.1 rgb color model for crt displays 100
4.2.2 subtractive color: cmy color model 101
4.2.3 transformation from rgb to cmy 101
4.2.4 undercolor removal: cmyk system 102
4.2.5 printer gamuts 102
4.3 color models in video 104
4.3.t video color transforms 104
4.3.2 yuv color model 104
4.3.3 yiq color model 105
4.3.4 ycbcr color model 107
4.4 further exploration 107
4.5 exercises 108
4.6 references 111
fundamental concepts in video 112
5.1 types of video signals 112
5.1.1 component video 112
5.1.2 composite video 113
5.1.3 s-video 113
5.2 analog video 113
5.2.1 ntscvideo 116
5.2.2 pal video 119
5.2.3 secamvideo 119
5.3 digital video 119
5.3.1 chroma subsampling 120
5.3.2 ccir standards for digital video 120
5.3.3 high definition tv (hdtv) 122
5.4 further exploration 124
5.5 exercises 124
5.6 references 125
basics of digital audio 126
6.1 digitization of sound 126
6.1.1 what is sound? 126
6.1.2 digitization 127
6.1.3 nyquist theorem 128
6.1.4 signal-to-noise ratio (snr) 131
6.1.5 signal-to-quantization-noise ratio (sqnr) 131
6.1.6 linear and nonlinear quantization 133
6.1.7 audio filtering 136
6.1.8 audio quality versus data rate 136
6.1.9 synthetic sounds 137
6.2 midi: musical instrument digital interface 139
6.2.1 midi overview 139
6.2.2 hardware aspects of midi 142
6.2.3 structure of midi messages 143
6.2.4 general midi 147
6.2.5 midi-to-wav conversion 147
6.3 quantization and transmission of audio 147
6.3.1 coding of audio 147
6.3.2 pulse code modulation 148
6.3.3 differential coding of audio 150
6.3.4 lossless predictive coding 151
6.3.5 dpcm 154
6.3.6 dm 157
6.3.7 adpcm 158
6.4 further exploration 159
6.5 exercises 160
6.6 references 163
ii multimedia data compression 165
7 lossless compression algorithms 167
7.1 introduction 167
7.2 basics of information theory 168
7.3 run-length coding 171
7.4 variable-length coding (vlc) 171
7.4.1 shannon-fano algorithm 171
7.4.2 huffman coding 173
7.4.3 adaptive huffman coding 176
7.5 dictionary-based coding 181
7.6 arithmetic coding 187
7.7 lossless image compression 191
7.7.1 differential coding of images 191
7.7.2 lossless jpeg 193
7.8 further exploration 194
7.9 exercises 195
7.10 references 197
lossy compression algorithms 199
8.1 introduction 199
8.2 distortion measures 199
8.3 the rate-distortion theory 200
8.4 quantization 200
8.4.1 uniform scalar quantization 201
8.4.2 nonuniform scalar quantization 204
8.4.3 vector quantization* 206
8.5 transform coding 207
8.5.1 discrete cosine transform (dct) 207
8.5.2 karhunen-loeve transform* 220
8.6 wavelet-based coding 222
8.6.1 introduction 222
8.6.2 continuous wavelet transform* 227
8.6.3 discrete wavelet transform* 230
8.7 wavelet packets 240
8.8 embedded zerotree of wavelet coefficients 241
8.8.1 the zerotree data structure 242
8.8.2 successive approximation quantization 244
8.8.3 ezw example 244
8.9 set partitioning in hierarchical trees (spiht) 247
8.10 further exploration 248
8.11 exercises 249
8.12 references 252
image compression standards 253
9.1 the jpeg standard 253
9.1.1 main steps in jpeg image compression 253
9.1.2 jpegmodes 262
9.1.3 a glance at the jpeg bitstream 265
9.2 the jpeg2000 standard 265
9.2.1 main steps of jpeg2000 image compression* 267
9.2.2 adapting ebcot to jpeg2000 275
9.2.3 region-of-interest coding 275
9.2.4 comparison of jpeg and jpeg2000 performance 277
9.3 the jpeg-ls standard 277
9.3.1 prediction 280
9.3.2 context determination 281
9.3.3 residual coding 281
9.3.4 near-lossless mode 281
9.4 bilevel image compression standards 282
9.4.1 the jbig standard 282
9.4.2 the jbig2 standard 282
9.5 further exploration 284
9.6 exercises 285
9.7 references 287
10 basic video compression techniques 288
10.1 introduction to video compression 288
10.2 video compression based on motion compensation 288
10.3 search for motion vectors 290
10.3.1 sequential search 290
10.3.2 2d logarithmic search 291
10.3.3 hierarchical search 293
10.4 h.261 295
10.4.1 intra-frame (i-frame) coding 297
10.4.2 inter-frame (p-frame) predictive coding 297
10.4.3 quantization in h.261 297
10.4.4 h.261 encoder and decoder 298
10.4.5 a glance at the h.261 video bitstream syntax 301
10.5 h.263 303
10.5.1 motion compensation in h.263 304
10.5.2 optional h.263 coding modes 305
10.5.3 h.263+ and h.263++ 307
10.6 further exploration 308
10.7 exercises 309
10.8 references 310
11 mpeg video coding i -- mpeg-1 and 2 312
11.1 overview 312
11.2 mpeg-1 312
11.2.1 motion compensation in mpeg-1 313
11.2.2 other major differences from h.261 315
11.2.3 mpeg-1 video bitstream 318
11.3 mpeg-2 319
11.3.1 supporting interlaced video 320
11.3.2 mpeg-2 scalabilities 323
11.3.3 other major differences from mpeg-1 329
11.4 further exploration 330
11.5 exercises 330
11.6 references 331
12 mpeg video coding ii m mpeg-4, 7, and beyond 332
12.1 overview of mpeg-4 332
12.2 object-based visual coding in mpeg-4 335
12.2.1 vop-based coding vs. frame-based coding 335
12.2.2 motion compensation 337
12.2.3 texture coding 341
12.2.4 shape coding 343
12.2.5 static texture coding 346
12.2.6 sprite coding 347
12.2.7 global motion compensation 348
12.3 synthetic object coding in mpeg-4 349
12.3.1 2d mesh object coding 349
12.3.2 3d model-based coding 354
12.4 mpeg-4 object types, profiles and levels 356
12.5 mpeg4 part10/h.264 357
12.5.1 core features 358
12.5.2 baseline profile features 360
12.5.3 main profile features 360
12.5.4 extended profile features 361
12.6 mpeg-7 361
12.6.1 descriptor (d) 363
12.6.2 description scheme (ds) 365
12.6.3 description definition language (ddl) 368
12.7 mpeg-21 369
12.8 further exploration 370
12.9 exercises 370
12.10 references 371
13 basic audio compression techniques 374
13.1 adpcm in speech coding 374
13.1.1 adpcm 374
13.2 g.726 adpcm 376
13.3 vocoders 378
13.3.1 phase insensitivity 378
13.3.2 channel vocoder 378
13.3.3 formant vocoder 380
13.3.4 linear predictive coding 380
13.3.5 celp 383
13.3.6 hybrid excitation vocoders* 389
13.4 further exploration 392
13.5 exercises 392
13.6 references 393
14 mpeg audio compression 395
14.1 psychoacoustics 395
14.1.1 equal-loudness relations 396
14.1.2 frequency masking 398
14.1.3 temporal masking 403
14.2 mpeg audio 405
14.2.1 mpeg layers 405
14.2.2 mpeg audio strategy 406
14.2.3 mpeg audio compression algorithm 407
14.2.4 mpeg-2 aac (advanced audio coding) 412
14.2.5 mpeg-4 audio 414
14.3 other commercial audio codecs 415
14.4 the future: mpeg-7 and mpeg-21 415
14.5 further exploration 416
14.6 exercises 416
14.7 references 417
ih multimedia communication and retrieval 419
15 computer and multimedia networks 421
15.1 basics of computer and multimedia networks 421
15.1.10si network layers 421
15.1.2 tcp/ipprotocols 422
15.2 multiplexing technologies 425
15.2.1 basics of multiplexing 425
15.2.2 integrated services digital network (isdn) 427
15.2.3 synchronous optical network (sonet) 428
15.2.4 asymmetric digital subscriber line (adsl) 429
15.3 lan and wan 430
15.3.1 local area networks (lans) 431
15.3.2 wide area networks (wans) 434
15.3.3 asynchronous transfer mode (atm) 435
15.3.4 gigabit and 10-gigabit ethernets 438
15.4 access networks 439
15.5 common peripheral interfaces 441
15.6 further exploration 441
15.7 exercises 442
15.8 references 442
16 multimedia network communications and applications 443
16.1 quality of multimedia data transmission 443
16.1.1 quality of service (qos) 443
16.1.2 qos for ip protocols 446
16.1.3 prioritized delivery 447
16.2 multimedia over ip 447
16.2.1 ip-multicast 447
16.2.2 rtp (real-time transport protocol) 449
16.2.3 real time control protocol (rtcp) 451
16.2.4 resource reservation protocol (rsvp) 451
16.2.5 real-time streaming protocol (rtsp) 453
16.2.6 internet telephony 455
16.3 multimedia over atm networks 459
16.3.1 video bitrates over atm 459
16.3.2 atm adaptation layer (aal) 460
16.3.3 mpeg-2 convergence to atm 461
16.3.4 mulficast over atm 462
16.4 transport of mpeg-4 462
16.4.1 dmifinmpeg-4 462
16.4.2 mpeg-4 over ip 463
16.5 media-on-demand (mod) 464
16.5.1 interactive tv (itv) and set-top box (stb) 464
16.5.2 broadcast schemes for video-on-demand 465
16.5.3 buffer management 472
16.6 further exploration 475
16.7 exercises 476
16.8 references 477
17 wireless networks 479
17.1 wireless networks 479
17.1.1 analog wireless networks 480
17.1.2 digital wireless networks 481
17.1.3 tdmaandgsm 481
17.1.4 spread spectrum and cdma 483
17.1.5 analysis of cdma 486
17.1.6 3g digital wireless networks 488
17.1.7 wireless lan (wlan) 492
17.2 radio propagation models 493
17.2.1 multipathfading 494
17.2.2 path loss 496
17.3 multimedia over wireless networks 496
17.3.1 synchronization loss 497
17.3.2 error resilient entropy coding 499
17.3.3 error concealment 501
17.3.4 forward error correction (fec) 503
17.3.5 trends in wireless interactive multimedia 506
17.4 further exploration 508
17.5 exercises 508
17.6 references 510
18 content-based retrieval in digital libraries 511
18.1 how should we retrieve images? 511
18.2 c-bird--a case study 513
18.2.1 c-bird gui 514
18.2.2 color histogram 514
18.2.3 color density 516
18.2.4 color layout 516
18.2.5 texture layout 517
18.2.6 search by illumination invariance 519
18.2.7 search by object model 520
18.3 synopsis of current image search systems 533
18.3.1 qbic 535
18.3.2 uc santa barbara search engines 536
18.3.3 berkeley digital library project 536
18.3.4 chabot 536
18.3.5 blobworld 537
18.3.6 columbia university image seekers 537
18.3.7 informedia 537
18.3.8 metaseek 537
18.3.9 photobook and foureyes 538
18.3.10 mars 538
18.3.11 vtrage 538
18.3.12 viper 538
18.3.13 visual retrievalware 538
18.4 relevance feedback 539
18.4.1 mars 539
18.4.2 ifind 541
18.5 quantifying results 541
18.6 querying on videos 542
18.7 querying on other formats 544
18.8 outlook for content-based retrieval 544
18.9 further exploration 545
18.10 exercises 546
18.11 references 547
index 551

.　　concepts.
　　Who Should Read This Book?
　　 This text aims at introducing the basic ideas in multimedia to an audience comfortable with
　　technical applications--that is, computer science and engineering students. It aims to cover
　　an upper-level undergraduate multimedia course but could also be used in more advanced
　　courses and would be a good reference for anyone, including those in industry, interested in
　　current multimedia technologies. Graduate students needing a solid grounding in materials
　　they may not have seen before would undoubtedly benefit from reading it.
　　 The text mainly presents concepts, not applications. A multimedia course, on the other
　　hand, teaches these concepts and tests them but also allows students to use coding and
　　presentation skills they already know to address problems in multimedia. The accompanying
　　web site shows some of the code for multimedia applications, along with some of the better
　　projects students have developed in such a course and other useful materials best presented
　　electronically.
　　 The ideas in the text drive the results shown in student projects. We assume the reader
　　knows how to program and is also completely comfortable learning yet another tool. Instead
　　of concentrating on tools, however, we emphasize what students do not already know.
　　Using the methods and ideas collected here, students are also able to learn more themselves,
　　sometimes in a job setting. It is not unusual for students who take the type of multimedia
　　course this text aims at to go on to jobs in a multimedia-related industry immediately after
　　their senior year, and sometimes before.
　　 The selection of material in the text addresses real issues these learners will face as soon
　　as they show up in the workplace. Some topics are simple but new to the students; some
　　are more complex but unavoidable in this emerging area.
　　 Have the Authors Used This Material in a Real Class?
　　 Since 1996, we have taught a third-year undergraduate course in multimedia systems based
　　on the introductory materials set out in this book. A one-semester course could very likely
　　not include all the material covered in this text, but we have usually managed to consider
　　a good many of the topics addressed and to mention a select number of issues in Part III
　　within that time frame.
　　 Over the same time period as an introduction to more advanced materials, we have also
　　taught a one-semester graduate-level course using notes covering topics similar to the ground
　　covered by this text. A fourth-year or graduate course would do well to consider material
　　from Parts I and II of the book and then some material from Part III, perhaps in conjunction
　　with some of the original research references included here and results presented at topical
　　conferences.
　　 We have attempted to fill both needs, concentrating on an undergraduate audience but
　　including more advanced material as well. Sections that can safely be omitted on a first
　　reading are marked with an asterisk.
　　What is Covered in This Text?
　　 In Part I, Multimedia Authoring and Data Representations, we introduce some of the no-
　　tions included in the term multimedia and look at its history as well as its present. Practi-
　　cally speaking, we carry out multimedia projects using software tools, so in addition to an
　　overview of these tools, we get down to some of the nuts and bolts of multimedia authoring.
　　Representing data is critical in multimedia, and we look at the most important data repre-
　　sentations for multimedia applications, examining image data, video data, and audio data in
　　detail. Since color is vitally important in multimedia programs, we see how this important
　　area impacts multimedia issues.
　　 In Part II, Multimedia Data Compression, we consider how we can make all this data
　　fly onto the screen and speakers. Data compression tums out to be an important enabling
　　technology that makes modem multimedia systems possible, so we look at lossless and lossy
　　compression methods. For the latter category, JPEG still-image compression standards,
　　including JPEG2000, are arguably the most important, so we consider these in detail. But
　　since a picture is worth a thousand words and video is worth more than a million words
　　per minute, we examine the ideas behind MPEG standards MPEG-1, MPEG-2, MPEG-4,
　　MPEG-7, and beyond. Separately, we consider some basic audio compression techniques
　　and take a look at MPEG Audio, including MP3.
　　 In Part III, Multimedia Communication and Retrieval, we consider the great demands
　　multimedia places on networks and systems. We go on to consider network technologies
　　and protocols that make interactive multimedia possible. Some of the applications discussed
　　include multimedia on demand, multimedia over IP, multimedia over ATM, and multimedia
　　over wireless networks. Content-based retrieval is a particularly important issue in digital
　　libraries and interactive multimedia, so we examine ideas and systems for this application
　　in some detail.
　　Textbook Web Site
　　The book's web site is www.cs.sfu.caJmmbook. There, you will find copies of figures from
　　the book, an errata sheet updated regularly, programs that help demonstrate concepts in the
　　text, and a dynamic set of links for the Further Exploration section of each chapter. Since
　　these links are regularly updated (and of course URLs change often) they are mostly online
　　rather than in the text.
　　Instructors' Resources
　　The main text web site has no ID and password, but access to sample student projects is
　　at the instructor's discretion and is password-protected. Prentice Hall also hosts a web site
　　containing Course Instructor resources for adopters of the text. These include an extensive
　　collection of online course notes, a one-semester course syllabus and calendar of events,
　　solutions for the exercises in the text, sample assignments and solutions, sample exams, and
　　extra exam questions.
　　Acknowledgements
　　We are most grateful to colleagues who generously gave of their time to review this text, and
　　we wish to express our thanks to Shu-Ching Chen, Edward Chang, Qianping Gu, Rachelle S.
　　Heller, Gongzhu Hu, S. N. Jayaram, Tiko Kameda, Xiaobo Li, Siwei Lu, Dennis Richards,
　　and Jacques Vaisey.
　　 The writing of this text has been greatly aided by a number of suggestions from present
　　and former colleagues and students. We would like to thank James Au, Chad Ciavarro,
　　Hao Jiang, Steven Kilthau, Michael King, Cheng Lu, Yi Sun, Dominic Szopa, Zinovi
　　Tauber, Malte von Ruden, Jian Wang, Jie Wei, Edward Yan, Yingchen Yang, Osmar Zaiane,
　　Wenbiao Zhang, and William Zhong for their assistance. As well, Mr. Ye Lu made great
　　contributions to Chapters 8 and 9 and his valiant efforts are particularly appreciated. We
　　are also most grateful for the students who generously made their course projects available
　　for instructional use for this book.