Pipeline Development

Kitsu

While at Winreality I was tasked with automating their video conversion process. I took it upon myself to create a video asset pipeline that would do the following primary steps.

• Ingest a new raw video file, query the database for the correct pitcher to match the file, collect pitch data from the user, create a new UUID asset, attach the video to the asset, upload the full package to AWS and move to the next step in the pipeline

• Asset would then attempt to be rotoscoped by in house ML engine created for this purpose. About 80% of frames were successful the remaining frames were sent out to be rotoscoped through Sagemaker

• Rotoscoped video is annotated with a tool written in Pyqt and using OpenCV to mark foot and hand locations

• Annotated video is re-encoded with FFMPEG

• Re-encoded video is then processed with After-effects to remove background and impose transparency

• Video is then contained in a unity asset bundle, pushed to aws, and the database is updated so it can find the new file.

This whole process was done by leveraging an opensource software called Kitsu which operates similarly to Shotgun (Shotgrid now). I created a linux server that was running the management and then created numerous python tools and automation jobs that would interact with kitsu to move jobs along or operate on them. The full process worked similarly to an animation pipeline except automated.