raptnrent.me

Making this week’s video was an experience. Being radio week, I had the idea to incorporate the DJ from The Warriors and wanted to use Wav2Lip to see how the syncing would work in a tight close-up. I used a generic synthetic voice for the audio because I didn’t think replicating the original voice would matter. The syncing script wouldn’t work though, because it said it couldn’t find a face to sync. Was it because it couldn’t find a nose and eyes to orient itself? In a previous video, the tool found non-face elements to sync, but maybe that happened because there had been a face in that part of the screen. So I opted for old-school dubbing, where they didn’t even try. It was nice that “boppers” almost lined up where it should though. For the music I used a Bob Ross mashup left over from a few semesters ago.

Then I wanted to bring Dr. Oblivion in, but found that he wasn’t answering his screen. It just went to static and stayed there. He’s since been found and is in good health, so don’t worry, but in the meantime I thought I’d enlist HAL-9000 from 2001: A Space Odyssey to act as a sub. I had previously cloned his voice because I had been thinking about staging a HAL vs KITT AI rap battle, and I wouldn’t have to worry about the lip syncing.

If HAL was really going to sub for the doctor, he should inhabit the TV. So I took a screenshot of the TV and put it in Photopea. I also found a GIF of HAL and brought that in as well. I checked the image sizes and made them both the same height. The HAL GIF was wider, but I could crop it later. I copied the TV and pasted it in as the top layer of the GIF, then erased the static on the screen so you could see HAL underneath. Since the TV was not an animated layer and it was on top, the animation, such as it was, would play underneath as if it was on the TV. Then I cropped it to the width of the TV layer and exported it as a GIF.

But I don’t think a GIF would work in iMovie, so I googled for a GIF to MP4 converter and used the first one that came up. I imported all the bits into iMovie – my part, the Max Headroom background, the Warriors DJ, the HAL MP4 and an MP3 I made with ElevenLabs. A nice thing about the MP3 is that the voice cloner picked up HAL’s background hum. The MP4 was only 4 seconds so I copied and pasted it a bunch of times. I had to split clips and move things around to get it in order for the video, and then I had to redo it because in all the confusion I left HAL with a lengthy stretch of dead air.

So it’s something, but naturally could be better. As much as I like HAL, he’s easier to take in shorter bits. The video could be more coherent, and certainly needs a better conclusion, but it’s a one-off that probably won’t get much play anyway. But it was a good experience to work  through because I get to see what works and what doesn’t, and I can use that in the future.

I ran into this error message when trying to run Wav2Lip last week:

I did file an appeal, asking what was suspected of being abusive, without expecting a response. Under their usage limitations they list deepfakes.

Anything I’ve made is too transparently faked to be considered deep, but some may argue otherwise. I wonder if it was Bob that busted me. Can’t trust those painter types. It got blocked in some countries due to copyright, so that’s why I’m suspicious. It’s better to work with public domain material anyway though.

I managed to do this with Frank and Vinny. So it should be possible with Grant and Hepburn. I’d have to down load the video, cut the parts before and after the phone conversation, and then separate the two sides to make training videos. After putting words in their mouths I’d have to piece it back together as a conversation. Seems tedious, but possible.