1 Topic by ben_elgert

  • ben_elgert
  • Member
  • Offline

Often videos are compared for a defined movement where the video playback speeds need to be adjusted for a more direct comparison either by reasons of: (1) video capture speeds that are unknown and different than playback speeds for one or both videos or (2) due to the compared movements being executed at slightly different speeds in real time. 

An example would be comparing baseball swings of a professional baseball player with an amateur player.  Say I am interested in comparing swings from load foot strike to ball contact on the bat.  The professional swing might simply be faster and all video capture and playback speeds are known but I want a direct comparison of that movement so I need to adjust playback speeds on one or both of the videos.  Or I might have a video of a professional player doing the movement (slow motion or full speed) at an unknown and different capture speed to that video’s playback speed and I need to adjust the playback speed on one of the videos to properly compare the movement.

Videos should be able to be synchronized on a particular frame with videos having different video capture and video speeds (specified or adjusted with Configure Video Timing…) and/or adjusted playback speeds on playback windows and have the same behavior when played using the common (synchronized) controls.  This is currently not the case with the current versions of the software.  Once playback speeds are adjusted to individually different values, the video that has finished first restarts and keeps playing a number of times and then pauses on the first frame until the other video has finished.  Also if you pause and restart the video synchronization, the video that will be the first to finish jumps back a number of frames for some unknown reason before restarting.

Principles:

- Video that needs to start first begins playing.
- Other video starts at calculated start time to match synchronization point.
- Video that ends first stops on last frame until other video ends at last frame
- Both videos reset to first frame once other video ends at last frame
- Video sequence is restarted if loop function is enabled or if play button activated

Adjustments to the individual video capture and video framerates or playback framerates using the speed control should not affect these principles.

Example:

Video 1:  500 frames@60 fps playback speed with speed setting at 1.00X, actual playback speed is 60fps, duration 8.33s
Video 2:  300 frames@30 fps playback speed with speed setting at 1.50X, actual playback speed is 45fps, duration 6.67s

Video 1: Synchronized on frame 180, time 3.000s
Video 2: Synchronized on frame 90, time 2.000s

Calculations determine that Video 1 must start first.  The delay in starting Video 2 = 3.000-2.000=1.000s.  Video 1 will have played 60 frames before the start of Video 2.  Video 2 will be at the last frame before Video 1 is finished at 7.67s (1.000s delay+6.67s duration) and should display the last video frame until Video 1 is ended and reset by the user or loop function.  Video 1 will be at frame 460 (7.67s x 60fps) when Video 1 is completed.

2 Reply by joan (edited by joan 2017-01-25 09:27:05)

  • joan
  • joan
  • Admin
  • Offline

I agree and I think it should work like this (except that the first-to-finish would wrap and wait on the first frame instead of the last because it was much easier to implement it this way :-)). Sometimes the algorithm is thrown back and misses the slot though.

The entire thing uses real time in the common slider, and translates real time to local video time (taking slow motion and capture speed into account) to seek to the correct point in each video. The correct behavior of this seek is unfortunately dependent on file formats. I have tested a few of these scenarios but yeah, definitely room for improvement in this regards.

For the scenario you describe you should unlock the speed sliders from the preferences in Playback > General. This will let you match durations of arbitrary sequences. This is requested more often than I anticipated. It seems this would be better as a button directly in the common controls UI instead of hidden in the preferences.

edit: I agree it is currently much harder than it should to perfectly match the duration of one performance in one video to the other, if you want to compare *forms* without regards for timing. As in your example of comparing two swings performed at different speed. Currently you need to tweak the slow motion in one of the video until it seems to cover the same duration.

A two-point synchronization system would mitigate that. You would mark the start and end of the movement in each video and the time would be remapped so that these sections are of the same duration in both. It would still be a linear mapping so the rest of the algorithm shouldn't change.

3 Reply by ben_elgert

  • ben_elgert
  • Member
  • Offline

Thanks Joan,

All the video comparison work I have tested has been with the speed sliders unlocked.  When the speed sliders have been adjusted differently from each other is when I started seeing some unpredictable things during synchronized video playback.  Most common is the video that finishes first restarts repeatedly instead of stopping until the other video finishes which makes things very confusing on the playback screen.

My next suggestion on synchronization also was going to be on a two point synchronization where the software would use two points defining the movement on each video to adjust both playback offset and playback framerate for one of the videos to compare the movements.  I have been sending other comments on the single point synchronization and didn't want to confuse the issue by bringing up two point synchronization but this is ultimately what I think would be the most useful.  The single point synchronization is a good start but you end up wanting to match the entire movement which requires adjustment in one or both of the playback speeds.  Adjusting the playback framerates for synchronization was the reason I brought up finer control of the speed sliders on the playback windows in another post.