Media Processing Workflow

Presentation Transcript

Media Processing Workflow Alex Zambelli Media Technology Evangelist Developer & Platform Evangelism alexzam@microsoft.com

Workshop Overview Silverlight media abilities Smooth Streaming 101 15 min break Live Smooth Streaming Workflow and engineering 15 min break Encoding for Smooth Streaming

A Brave New World Encoding 320x240 video on a portable PC is "so 2002" HTTP versatile gushing has totally changed the amusement in the previous 2 years The future for Silverlight is all Smooth Streaming, all HD Video quality is better than anyone might have expected, yet at the cost of intricacy and data transmission Don't stress, it just gets less demanding from here

Silverlight Media Formats Windows Media ASF document organize (*.asf, *.wmv, *.wma) VC-1 ( WMV9 ), WMV8 , WMV7 video codecs WMA Pro, WMA Standard, MP3 sound codecs MPEG-4 MP4 record design (*.mp4, *. 3gp , *.mov) H.264 video codec (Baseline, Main, High profiles) AAC sound codec ( AAC - LC profile)

Silverlight Media Extensibility Smooth Streaming VC-1 and H.264 video WMA Pro, WMA Standard and AAC - LC sound Supported by means of Smooth Streaming Media Element expansion MediaStreamSource gives designers a chance to include bolster for 3 rd party codecs , arrangements and conveyance techniques

Silverlight Media Protection Silverlight DRM , controlled by PlayReady AES-256 encryption Windows Media DRM encryption Silverlight 3 VC-1 and WMA encryption just Live DRM (Direct License Acquisition) just Silverlight 4 H.264 and AAC encryption Offline DRM

Silverlight Media Delivery Windows Media Services spilling Windows Server 2008 and 2003 Unicast ( WMS HTTP) Multicast HTTP dynamic download Any HTTP 1.0/1.1 web server IIS Smooth Streaming Windows Server 2008 ( IIS7 )

Silverlight Media Framework No compelling reason to reexamine the wheel each time a Silverlight video player is required Silverlight Media Framework: http://smf.codeplex.com All basic playback highlights in addition to cutting edge Smooth Streaming components DVR, Fast Forward, Rewind, Replay, Slow Motion Markers, inscriptions, dynamic promotion inclusion Rich investigation and logging

Media Delivery Methods Unicast Traditional Streaming Progressive Download HTTP-based Adaptive Streaming Multicast IP Multicast

Unicast Many balanced streams Requires more data transfer capacity per client May require more servers For private and open systems Origin Server

Multicast A solitary one-to-many stream Uses transmission capacity of just a single stream Requires multicast-empowered systems Typically requires less servers Origin Server

Unicast Methods Movie HTTP Adaptive Streaming Video @ 01: 04 ? Video @ 01: 06 ? Video @ 01: 08 ? Customary Streaming Play Seek Pause Progressive Download Movie

Internet Streaming Challenge Traditional gushing intended for productive media conveyance Works well in little systems yet on the Internet experiences genuine downsides: Doesn't scale to today's Internet group of onlookers Doesn't consider today's Internet structure and association Designed for system conditions less unstable than Internet movement

HTTP Adaptive Streaming Hybrid media conveyance technique Acts like spilling yet is in actuality a progression of short dynamic downloads Video and sound conveyed as arrangement of little records over HTTP Built for the Web Leverages existing HTTP stores Scales especially well to take care of popularity Resilient to network flightiness Client can flawlessly switch video quality and bit rate in light of saw system data transmission and CPU assets Applicable to both on-request and live conveyance

Windows Server Media Delivery IIS Media Services Traditional Streaming Unicast WMS RTSP WMS HTTP Multicast WMS Multicast Progressive Download Bit Rate Throttling HTTP Adaptive Streaming Smooth Streaming

Smooth Streaming Smooth Streaming Microsoft execution of HTTP-based versatile gushing Best of both universes Responsive and effective like spilling Cheap and adaptable like dynamic download Superior client encounter No buffering, no stammer Instant begin, moment look for Seamless piece rate exchanging in light of transfer speed and CPU

Multi Bit Rate Revisited 00:27.24 03:47.16 05:05.04 07:33.10 2.4 Mbps 1.5 Mbps 700 kbps 300 kbps

Adapting Bit Rate in Real-Time 300K @ 00: 00 ? 700K @ 00: 02 ? 2.4M @ 00: 04 ? 1.5M @ 00: 06 ? 2.4M @ 00: 08 ? 300K (begin rapidly) 00: 00: 02 00: 04 00: 06 00: 08 700K (great system) 2.4M (extraordinary system) 2.4M 1.5M (glitch) 2.4M (play on… ) … Bit Rate Heuristics 300K

Smooth Streaming Design Smooth Streaming File Format in view of MP4 (ISO Base Media File Format) Video is encoded and put away on plate as one bordering MP4 document One record for every piece rate Each video Group of Pictures (GOP) is put away in a different Movie Fragment box This permits exact discontinuity at key casings Contiguous record is essentially part up into lumps when reacting to a customer ask for

Smooth Streaming File Format File Type ( ftyp )

On-Demand Presentations MP4 documents containing video/sound/information sections New document expansions rather than *.mp4 *. ismv for documents containing video and sound tracks, or just video tracks *. isma for documents containing just sound tracks On-request server show record File expansion: *.ism Describes the connections between media tracks, bitrates and documents on circle Uses SMIL 2.0 XML design Client show record File augmentation: *. ismc Describes accessible streams, codecs , bitrates, resolutions, metadata Uses XML design

On Demand Server Manifest

On Demand Client Manifest

Client-Server Communication All solicitations to the server are shaped as RESTful URLs Client constantly first demands the customer show: http://iis.net/video.ism/Manifest After parsing the show the customer starts making demands for pieces by demonstrating bit rates and balance times it gained from the show: http://iis.net/video.ism/QualityLevels(350000)/Fragments(video=40040000) http://iis.net/video.ism/QualityLevels(48000)/Fragments(audio=62293333) Every substantial HTTP ask for returns a document of MIME sort "video/mp4"

Serving Fragments On Demand IIS7 Smooth Streaming handler deciphers the RESTful URL ask for It maps the quality level to a physical ISMV record in light of the server show (*.ism) and ascertains the part area inside the touching ISMV record in view of the time counterbalance Because the wire organize (section) is only a subset of the circle arrange (MP4), the extraction procedure is straightforward No re-muxing important Dynamic stream exchanging rationale is completely executed in Silverlight application code – no server-side location

DEMO: Setting up On Demand distributing guides Using Fiddler toward screen Smooth Streaming activity

Live Smooth Streaming Live distributing point distinguished by *. isml expansion Client-server correspondence fundamentally the same as on-request Server constantly affixss MP4 record with new pieces from encoder and makes them accessible to customer This permits DVR-like looking for inside the current file

Encoder-Server Communication Encoder pushes divided MP4 stream to server in body of a long-running HTTP POST ask for http://iis.net/live.isml/stream(720p) Encoder embeds Live Server Manifest Box into begin of MP4 stream Contains a SMIL-arranged Live Server Manifest fundamentally the same as On-Demand Manifest Describes all tracks in that encoder's stream

Encoder-Server Communication Each sent section contains a container with supreme time and term Timestamps are commonly gotten from the information source Linear Time Code (LTC) If numerous encoders are utilized to encode a similar video source, it's basic that a period code generator is utilized to keep their LTC in a state of harmony

Serving Live Fragments Server parses the encoder shows and begins gathering MP4 parts Server manufactures an aggregate runtime file in memory for every single approaching part Server stores parts into a transitory DVR ISMV or a lasting ISMV document When the bar point is closed down, server produces " mfra " file box for ISMV chronicle

Live Client Workflow Client asks for the most recent show http://iis.net/live.isml/Manifest Server includes all the encoder shows and the runtime piece list and sends the most recent variant to the customer Every part contains data about the following section No compelling reason to over and over download show Client assembles its own total runtime file in view of the lookahead data in the pieces

DEMO: Basic Live Smooth Streaming work process

Live Smooth Publishing Points ISML documents are SMIL 2.0 designed XML setup records Live distributing focuses can be made through the IIS Manager UI or by physically making ISML documents on server Both push and force distributing models are upheld Live distributing focuses can likewise push to and pull from other live distributing focuses

Live Publishing Point – IIS Mgr

Live Publishing Point - ISML

ISML Syntax sourceType Push : source streams will be pushed to bar point Pull : bar point will pull streams from recorded URLs distributing Streams : bar indicate can serve streams other live bar focuses Fragments : customers can ask for pieces Archives : filing is empowered lookaheadChunks : number of sections server will cradle before making them accessible to customers manifestWindowLength : length of DVR moving window archivePath : nearby way where files will be composed

Ingests and Origins Origin Server: Delayed Stream Origin Server: Delayed Stream Encoder Ingest Server Origin Server: Realtime Stream Origin Server: Realtime Stream 8 Mbps Origin Server: VOD Archive Origin Server: VOD Archive

DEMO: Advanced Live Smooth Strea