Folder Layout pt. 2

Keeping your socks in your boots

It may be very beneficial to ensure that all data related to any other data are kept together. Like sequences can contain shots, a shot may contain more than simply the users working directories, such as its description, length, padding, camera information, frame rate, media such as video-references for animators or audio for lipsync.

Tools already exist for this purpose. Some examples I already mentioned in the previous post but I'll repeat them here for completeness. 

Shotgun, FTrack, Tactic. 

I really recommend checking those out, they've got videos showing some fundamental functionality and they are used today by hundreds of studios. I'll refer to these tools as Offline Asset Management, or OFAM for short. They provide two major concepts rolled into one - tracking assets and shots along with assigning tasks and maintaining schedules of projects. The tracking part enable users to associate data that would otherwise be difficult to associate via a simple hierarchy of files and folders, along with the ability to add links other than simple hierarchical ones, such as their relationships to users or even other data in the database. They do this by separating the binary data from its related information.

Separating binary data from related information

The caveat of this approach however is that it is external to the files they augment. Augmenting data, aka Metadata, is stored separately from their associated assets, shots and sequences in a database table such as SQL.

This means that whenever a change happens on one side, the other side needs to get updated somehow. This introduces the concept of syncing and is generally handled like this - a change in Maya, say a publish of a character, will trigger an event (referred to as a callback) which performs the corresponding action on the server. Data generally only travels in this one direction as any change in any of an objects metadata, such as shot description, has no relevance within, in this case, Maya (or does it? I'll dive into this more later!). For example, a new asset is created from Maya. Maya triggers a callback and the equivalent asset is created within the OFAM.

Psuedo-code of mirrored command

Some data will remain unique on either end, such as the binary scene file, while other data, such as camera information, will overlap and cause duplicity.

Overlapping data means duplicated data

In order to store the camera information of a shot, the shot itself and all of its related hierarchy must exist within the database, if only to host that information, so that users have a way to associate it and to search for it. However the hierarchy must also exist to host the binary files.

While the overlap is very small in terms of megabytes on disk, we aren't concerned with disk space issues. What matters is the information they represent and our ability to efficiently find it.

The overlap applies to both shots, sequences and assets and also any media associated. Some media are only accessible via the OFAM, such as a reference material, thumbnails, playblasts etc., but must yet be stored in binary form somewhere on disk. This means that you end up having an additional hierarchy of binary files, besides the original binary files themselves.

Superfluous complementing data