Audio podcast 2 - the engineering!

Hugh and I continue our discussion of audio and make particular mention of cabling for TV facilities.
Find it on iTunes, vanilla RSS, YouTube or the show notes website.

The next podcast - Audio 101

Hugh and I go over the fundementals of sound and recorded audio in the first of a two-parter Find it on iTunes, vanilla RSS, YouTube or the show notes website.

Crimp tool for all current video ends?

All the wiremen want me to buy them one of these! This tool can do SD01 standard def coax (Image 360), 1694 HD Coax and the newer Din1.0/2.3 mini-HD connectors.

UK electricity demand

After a major power-outage in North London this afternoon I went looking at the National Grid website; there is some fantastic realtime data there about the amount of power the UK is consuming and how much juice is transferred between regions.
I nicked these two applets but click the title link and have a truffle around - 41GWatts currently(!)

Here's an update - interesting article from The Independent on 13th April;

How the Supergrid could help keep the lights on about the electrical interconnects between the UK and other countries; apparently a 900-mile 0.8GWatt link is being proposed to Iceland!

Reliability of modern hard drives

A couple of things have got me thinking about the reliability of hard drives.

1. The eldest boy asked me about RAID levels and what extra reliability they bring.
2. I sold an old 500gig SATA drive from my media center to a guy on eBay and he's quibbling because he's found three bad sectors

Although RAID has bought tremendous increases in the speed and reliability of storage systems there are some basic engineering considerations around combining many unreliable parts into a whole. The Mean Time Between Failure for modern hard-drives (MTBF) is in the order of 30,000 hours. MTBF is a complicated field but if you look at the figures provided by manufacturers then they assume a Gaussian distribution with the 30,000 hours figure at the peak; a few drives will fail after a day, a few will last 100,000 hours, but the bulk will fail around 30,000 hours (around three and a half years). It's why I say to people "...it's not if this drive will fail, rather it's when!"
So - with this in mind I decided to find out the MTBF of a rack of ten drives, each with a MTBF of 30k hours.The formula is;

So, if you stick 30k hours in for D1 through D10 you find the MTBF for the entire system is only 3k hours - less than twenty weeks! In fact it's worse than this as the PSU in the enclosure and the RAID management card will have MTBFs to take into consideration as well. In fact if you ask any broadcast engineer "how often are you replacing drives in RAID arrays" and they'll tell you it's nearly a weekly occurrence for any decent sized facility, and this is why! Although it's been nearly a decade since I ran engineering in a good-sized facility I was often uneasy about how often RAID enclosures failed (loosing all of the media, which is what happens with a RAID-0 striped set). I had that 30k hours figure in my head but never calculated the system MTBF.

RAID-1 (mirrored drive sets), RAID-5 (distributed parity) and RAID-6 (double distributed parity) along with some of the advances that better file systems bring (Isilon's OneFS and Linux's ZFS) mean that failure of a single drive is no longer the disaster it once was, but if someone doesn't notice a drive has died OR (heaven forbid) a second drive dies whilst replacing the first you're stuck. Don't forget a lot of the chassis we're installing now have a couple of dozen drives (Isilon's NL36 nodes - done of a few of them recently) and so MTBF is even worse than the 3k hours above (however, server-grade SAS or Fibre Channel drives are considerably more reliable than domestic-grade SATA drives).

We also know that modern multi-terrabyte drives pack data so densely (similar sized platters to the first 10Mbyte drives of yesteryear, but hundreds of thousands more bits/mm-sq. of disk surface) that the disk's error correction/error recovery system is working flat-out all the time. The newer 2Tbyte drives have a Viterbi decoder to try and statistically extract correct data from the very noisy signal coming off the drive's heads. Additionally the drive's SMART system has to know about the number of bad sectors due to manufacturing imperfections (contained in an EPROM-based p-list table) as well as the number of grown bad sectors (which get swapped out as per the g-list). Spinrite is the best utility I've found for drive maintenance/recovery as it forces the SMART system to pay attention to bad sectors and swap them out. In a Hitachi Ultrastar 7k RPM, 500gig SATA drive there are 10,000 hidden spare sectors on the drive (each sector is only 4k bytes in size) to allow the drive to swap-out failed sectors. According to the data sheet Hitachi would replace a new drive if it had more than twenty bad sectors from the factory - any less and they regard it as being well inside manufacturing tolerances. If you Google "how many bad sectors is acceptable for a new drive" you'll find hundred of IT experts claiming that no bad sectors are acceptable. I don't know what planet they live on, presumably one where quantum mechanics operates in a different manner and electrons don't bump into each other leading to electrical noise!
Oh - the eBay guy; he ran a utility on the drive I sold him that reported three bad sectors. He asked me for a refund. Apparently a second-hand disk drive should carry a better guarantee than that provided by the factory when new!

Colourimetry 2 - Calibrating monitors for TV

After the intro to colourimetry Hugh and I talk about calibrating monitors for film and TV use. Find it on iTunes, vanilla RSS, YouTube or the show notes website.

TV Colour - the next podcast

Hugh and I lay the groundwork for the next podcast about monitor calibration. This episode concerns perception of colour in TV. Find it on iTunes, vanilla RSS, YouTube or the show notes website.

Speeding up HTTP

The big overhead with protocols that run on top of TCP/IP is the number of connections they open. A modern web page has many different kind of assets (HTML, scripts, Java, GIFs, JPEGs, etc etc) from many places (the primary domain, adservers, Google+ buttons etc etc) such that when you load the front page of facebook.com you may well have opened and closed a thousand TCP connections to a dozen servers. It's amazing that it works at all, but when you consider that each connection had to not only do the three-way TCP handshake, but it also had to run TCP flow-control starting slowly and ramping up packet rate until it started to drop packets and then dropping back. When Mr Burners-Lee wrote his original http server in the early nineties there is no way he could have anticipated how the web would grow.

It seems there are several approaches to optimising this - for the past ten years there have been various attempts to re-define TCP; MUX, SMUX, and SST protocols were valiant efforts that died on the vine because they essentially break how the infrastructure works. Whatever comes after http has to work over all the same IP v.4 routers, switches and proxies. In the last year I've become aware of two projects that work and don't require funky routers etc.

1. Amazon Silk - this is embodied in the browser that comes on the colour Kindle. Essentially it is a mega-proxy that not only collects all the assets for the page but re-renders pictures etc for the smaller screen and sends the whole lot in a stream. So one connection allows the whole pre-rendered page to arrive with the assets from Double-Click and Google (and any other third-party elements in the page) pre-collected for you by Amazon. It runs of their EC2 platform and does depend on them providing the service.
2. SPDY is a Google-sponsored project that doesn't replace http but optimises it. By employing pipelining (i.e. keeping a TCP connection open for all the assets from one domain), compressing headers and marking those headers that don't change so they don't need to be re-sent it speeds up the browser by around three-fold. Further speed increases come if the web-server is able to collect the third-party assets and deliver them over the same pipeline.

As ever Steve Gibson has covered these very comprehensively in Security Now! - SN320 for Silk and SN343 for SPDY.

What's in your rucksack?

Phil and High go over the basic set of tools and test kit that broadcast engineers on the hoof need! Find it on iTunes, vanilla RSS, YouTube or the show notes website.

Synchronous ain't always best!

Engineers have been trained since time immemorial that with multiple sources of video the best thing is that they are always locked together and timed to a common reference - ideally station black & burst (or TriSyncs nowadays). The reasons for this are numerous, but a couple;

1. Studios cameras into a vision mixer have to be locked to achieve clean cuts between the cameras - it would look rubbish if you had a frame roll every time you cut between sources. The same is true of sources into a continuity suite etc.
2. In the case of Avid for the longest time you had to make sure the VTR was locked to the same reference as the Media Composer (all the way from v.7 ABVB systems in the mid-90s to the last revision of Adrenaline in 2007!) otherwise the audio and video capture portions of the machine would free-run WRT each other and within minutes you'd be loosing frames of sync.

So - I had a coffee and chat with a pal this morning who works for a big facility that has won an archive digitising project and they are using the BlackMagic DeckLink 4K card to allow them to ingest 4 x DigiBeta tapes at once. The capture software is ToolsOnAir and they found that after the first clip was captured the second clip would be a frame out of sync, and progressively worse after that - unless you re-booted between captures! It turns out that if the four VTRs are allowed to free-run then you don't get the problem. Perhaps processing the vertical syncs places a burden on the card/software and if it happens simultaneously on all four inputs trouble ensues?!

It reminded me of a situation with a big broadcaster who was distributing their regional variations over Astra on a single multiplex. The stat-mux was very unhappy with material that was (for the most part) identical; only the ad-breaks differed. Most video cuts occurred at the same time across all six SDi feeds. The solution was to apply a two frame delay between all the sources (so o/p 6 was now 12 frames late WRT o/p 1).

Mains 101 - part 2 / Postscript

Hugh and Phil wrap up the previous mains safety podcast with some corrections and photos from Nigeria! Find it on iTunes, vanilla RSS, YouTube or the show notes website.

Phil & Hugh's next podcast - electrical safety.

Find it on iTunes, vanilla RSS, YouTube or the show notes website.

I'm representin' at BVE

Today I'm delivering a 'training taster' session at BVE 2012. Oh, the title link is my notes if you want to download them.

Phil & Hugh's first podcast - Fibre 101; what every broadcast engineer should know

I've been bugging various industry friends to join me in an engineer's focused podcast for a few years now. Only my pal Hugh Waters (@hugh_waters on Twitter) stepped up and we have a whole series planned in this format.

Coming soon; Mains & Electrical Safety as well as TV Colourimetry.

Find it on iTunes, vanilla RSS, YouTube or the show notes website.

AES pinouts for old Sony PCM800 recorder

Struggled to find this the other day.

Re-formatting PDFs to read nicely on Kindle

The Kindle is a great little machine (and the £89 keyboard-less version is excellent value). If you buy books or newspapers from Amazon they are perfectly formatted for the 6" screen and arrive in the proprietary file format. However, most of us have a stack of PDFs (books, work material etc) that you can drop into the /documents folder on the Kindle, but if they're formatted for A4 printing then you're either looking at very small text or trying to zoom & scroll the display.

k2PDFopt does a very good job or re-flowing the pages of a PDF (keeping diagrams in the correct place) to fit the 6" screen. I haven't yet tried the Windows or Linux versions but the Mac build is a bit fiddly (make sure you read the install instructions!) and you can't be shy of the command line!

My home AV rig

I'm often asked what I use at home for music/TV/movies - nothing fancy (haven't got a 5.1 rig yet!) because money has been tight for the last few years and I try and follow the ex-work/cheap-on-eBay way of getting things done. As a family we all listen to podcasts and music on 'phones/iPods/USB-stick-in-car and since we don't have any consistent manufacturer for anything we keep everything as MP3s and DivX AVIs or MPEG2 (for off air recordings of TV shows).

• Music - I started encoding my music in 1999 when I got my first MP3 player (the mighty Diamond Rio 500 in case you remember!) and so experimented with different encoders and data rates. At the time I concluded that the Fraunhofer MP3 encoder at 128kBits/sec was adequate if I was using ear-buds or in the car. I quickly changed my mind and have been encoding all my music at 192kBits/sec using variable bit-rate. I have blind-tested myself on good speakers and conclude that for my middle-aged hearing I can't spot the difference between uncompressed and that data rate. MP3 is the way to go as I have a mix of playback devices (iPhone for personal listening, Dell Digital Audio receiver for the living room along with various PCs & other-brand 'phones & MP3 players for the rest of the family). Consequently I always set iTunes to encode to MP3;
  
  
  I often hear people banging on about FLAC and Apple Lossless but I'm not that bothered. I tend to listen to music for the lyrics and chord progressions, not that last 0.01% of perceived fidelity. I spend my life listening to proper speakers (i.e. thousands of pounds a pair) either from behind a mixing desk or in a dubbing suite so don't try impress me with what you bought in the high-street! I know what good audio sounds like.


• TV recording and video playback - I'm using Windows 7 on a ten year-old workstation to make recordings off air; two £15 no-name Maplin-special USB DVB-T sticks allow multiple recordings via Windows Media Centre. I keep thinking I'll buy a DVB-T2 stick so I can make FreeviewHD recordings but I haven't yet, partly because iPlayerHD is so good. The machine that feeds the living room TV is the same machine machine that is the kitchen iTunes/radio machine - it has two soundcards and dual-display. In there kitchen there is a mini keyboard and mouse and in the living room a hand-held RF mouse. You wouldn't know it was the same computer save for when you want to fast-forward the video playback whilst someone in the kitchen is trying to find a song in iTunes and the mouse pointer jumps off your screen!


• Commercial removal/editing - I used to be a big fan of ComSkip as it is an excellent automated ad-break remover for Transport Stream video. However, I watch so little commercial television that I've found my regular MPEG editor Video Redo to be just as good;
  
• DVD / BluRay - I grabbed a Sony S370 on eBay for £70 and it is an excellent machine; actually better than the S380 that replaced it (Sony lost the right to several codecs). It supports several online video services.
• iPlayer - we used to use the Wii as an iPlayer machine and it is really good, but standard definition only. In fact I picked up a Wii with a faulty DVD drive for my Mum and loaded the iPlayer client onto it and she uses it as her BBC on demand machine. We now use the Sony BluRay player as it supports HD iPlayer and the pictures at 5mBits/sec are indistinguishable from off-air HD playback. Even the SD content looks better than the Wii's output.
  

So that's it - I'm not in a position to spend thousands of pounds but I am pleased the way I've got it all working for virtually no money.

Subtitling software and my new Bluray player

Occasionally my PVR makes an incomplete recording of a film that Sarah and I wanted to watch. It was the case last week with the Italian film Couscous and so I didn't feel too bad snagging a copy from a torrent site. However, when we sat down to watch it I realised there were no subtitles (either burned into the video or as a separate text file). "Not a problem" I thought - straight to one of the many sites that carry subtitles for every film ever released and I grabbed a likely looking .sit file for the movie. After discovering that Microsoft have pretty much removed all subtitle support from Windows 7 (my PVR is MediaCentre) I tried watching it with that old faithful backup VLC player (an order of magnitude better than WMP12 in every respect!) - a good example of where open-source software makes the closed-source equivalent look very silly - but I discovered the subtitles in the file were a consistent twelve seconds late. I suppose the subtitler was working off a different version of the film, maybe he started his frame count with all the film and distribution company bumpers?

Again, no worries, I fired up my previous standby for manipulating subtitle files DivXLand Media Subtitler only to discover it can't slip ever sub in a file by a defined amount. In every other respect it s an excellent utility handling twenty-odd file formats and having auto-timing functionality as well as individual sub-sync features, but couldn't handle this problem easily.

So, bit of Googling revealed another free and excellent utility Subtitle Edit which offers pretty much the same toolset but with the ability to slip the sync on groups of subs. Just what I needed; it has a better preview facility as well so you can drop into various places of the video file to check the captions are consistently running to time and it will automatically pull captions a few frames either way when the audio waveform doesn't quite match with the start frame-code of the subtitle.

After this VLC played the .avi & .sit combo perfectly but not WMP12 or MediaCentre. At that point my thoughts turned to my new (2nd hand!) Bluray player; a Sony BDP-S370 which we've had for a week (£70 on the eBay) and have been mightily impressed with it's network video functionality. It's the best iPlayer machine I've found so far bar none (much better than the Wii, Virgin Media & Tivo) and it will happily play the file and display the subtitles either off a thumb-drive or via the network using DNLA.

RS422 / Sony P2 protocol and serial stuff!

When running P2 protocol over RS422 (i.e. Sony VTR remotes) there is no hardware handshaking so RTS and CTS (Request To Send and Clear To Send) aren’t used; a bit like the old 3-wire XModem/YModem/Kermit protocols used in RS232 (remember RS422 is just a balanced version of RS232).

We base our RS422 wiring on the Quartz remote standard (Quartz were one of the first firms to use RJ45s & cat5 for RS422 remotes):

However – I know for certain that Probel use a different standard and many places are wired to whatever the local standard is; remember – until ten years ago most places wired ‘422 on star-quad cable rather than cat5e/6. I don’t know if current model Evertz routers have maintained the Quartz standard – I bet they have given they bought Quartz for its router business.

Whatever wiring standard is used always make sure that pins 2 & 7 are a twisted pair and likewise 3 & 8 otherwise you lose all the advantage of common-mode noise rejection that balanced RS422 brings.

Finally you need to be certain if a place is wired for chassis earth (pin 1 on a 9-pin) or signal earth (pins 4 & 6 on 9-pin). Signal earth is best as there is always a chance of earth-hum between areas when you tie chassis earths together but hopefully properly designed kit with balanced lines have the signal earth floating WRT to power/chassis ground. BUT, you have to stick with the local standard; if the engineer has wired only chassis earths you need to continue using pin 1 or even shorting pins 1, 4 & 6 at the remote end.

Remote control options

If (like me) you find yourself as the default tech support provider for friends and family you've no doubt wondered about remote desktop software - VNC, RDP, Apple remote desktop, or any of the paid-for managed services (Go To Assist, LogMeIn etc).

I think there are several things to bear in mind;

1. NAT routers in the way? If you're merely using remote desktop to go between machines on the same LAN then this isn't an issue but if you have to take control of your Mum's laptop and you're both behind routers then you either have to have made a hole in her's or be using a protocol that supports NAT translation.
2. IP address - again, the person you're trying to reach may well be on a dynamically assigned IP address.
3. Bitmap vs remote GUI rendering; VNC sends a bitmap (admittedly compressed) and so maybe sluggish whereas Windows RDP or Apple remote desktop send GUI primatives which render at the remote end.
   
4. What combination of OSes are you using? Running Windows but supporting someone on a Mac? The remote desktop client built into OS-X since Tiger falls back to VNC if the remote machine isn't a Mac - nice touch.

So - in the case of my Father-in-law's Windows XP desktop machine I use VNC every time - This is because I don't know if he's going to call me during the working day (when I'm using an OS-X laptop) or in the evening when I'm likely on a Windows 7 or XP desktop. Since his machine is fixed I had the liberty of installing a DynDNS account on his router (so I hit a username.dyndns.org address rather than trying to discover his internet-facing IP address) and I opened a hole in his routers firewall (so traffic on TCP port 5900 gets mapped through to his PC). With all that in place I know I can grab control of his desktop using TightVNC (my favorite VNC client) under Windows or the built-in remote desktop of Snow Leopard;

On the other hand my Mum has a laptop which may or may not be at her house. Since she is running Windows 7 and I can always get to a Win7 machine she Instant Messages me with a Windows RDP support request and after a bit of typing in confirming codes it works well without having to worry about IP addresses or NAT traversal.

That leaves the paid-for server-based systems like Log Me In and Go To Assist which require no software installed (it's done via a quick Java download) and take care of NAT traversal etc.

So - you pays for money, you takes your choice. I prefer VNC because it's open and works across OSes. It does require a bit of work to send it across the public internet. After that Windows RDP is fine if you have contemporary Windows boxes. I suspect at some point I'll sign up to Go To Assist and pay as it is very convenient and works entirely well across networks and OSes.

VNC connected to my home Windows 7 media machine, running inside a Windows 7 virtual machine on my Macbook Pro under OS-X