$ sudo apt-get update
$ sudo apt-get install libtheora-bin 

libtheora-bin is installed in your system.

Make ensure the libtheora-bin package were installed using the commands given below,

$ sudo dpkg-query -l | grep libtheora-bin *

You will get with libtheora-bin package name, version, architecture and description in a table

libtheora-bin package basic information: