[jitter] cc.uyvy2rgba.jxs and optimization

Hm. Unless I am mistaken, UYVY does absolutely nothing with the alpha  
channel what so ever. That information is lost to the ether unless you  
use the alphaglue shader and read in the alpha via other means. UYVY  
has no alpha channel, and I dont think that jit.qt.movie premulitplies  
alpha for you while doing the conversion?

 From my understanding of the UYVY shaders and colormode is:

*Some video codecs use an internal format that is something like ARGB  
(or RGBA...), or YUV or some variant (Y, cr' cb' or whatnot)

*jit.qt.movie when left alone, will always do conversion to JITTERS  
basic ARGB matrix format.

This means that any video whose codecs native internal pixel format is  
YUV or some other needs to be converted to ARGB by either quicktime or  
ji.qt.movie (I am unsure which internal mechanism does this). If the  
pixel format is something like RGBA or ARGB, nothing really needs to  
be done.

jit.qt.movie @colormode uyvy does the reverse of this, which means any  
quicktime movie whose normal internal pixel format is YUV (or  
similarly compatible) gets left alone, otherwise it is converted to  
UYVY.

This means that for codecs which normally use YUV or whatnot  
colorspace internally, using @colormode UYVY is a fastpath - as there  
is no conversion overhead.

This also means that ARGB codecs (or anything else), using @colormode  
UYVY is NOT a fastpath, as conversion must happen (somewhere - again I  
do not know specifically where this happens).

Id love to know from Cycling if this is correct.

*feature request below!*

Ive been thinking about this myself, and it might be nice if there was  
an attribute @colorspace native

which would also dumpout the colorspace on the rightmost outlet  
(dumpout outlet) right before  the movie plays its first frame, and  
load the movie natively.

This would allow you to :

Have a fastpath GPU uploading solution that would retain alpha channel  
should your quicktime movie have one, or skip right to UYVY those that  
need it.

decide what to do with the alpha channel should you want to keep it.


* As far as the errors go, have you tried the other various uyvy  
conversion shaders, the lite, and the one with gamma correction?

As far as animation is concerned I belive it is

ARGB and uses only run length encoding, which is basically like  
uncompressed, and has not inter frame coding. Its like motion jpeg/ 
photo jpeg, just ARGB and lossless compression.


On Jan 2, 2008, at 7:09 AM, John Dekron wrote:

> Hi _n,
> I get the digital artifacts as well and have no idea where they come  
> from.
> Animation and png are videocodecs that hold an alpha channel what is  
> necessary for the uyvy information. If I remember correctly it is  
> possible to limit the data rate on png compression. I did use png  
> for a previous project, but lately I experienced some crashes when  
> playing back png compressed movies. i switched to animation and it  
> worked fine. Sorry that I didn't investigate deeper on the crashes  
> that I had, so this information might as well be treated as gossip-  
> but eventually there is some truth in it.
>
> good luck. John.
>
> (())_n schrieb:
>> yes dekron, yair,
>> I have been looking into this method of doing the conversion  
>> before, and it does help, doubling the frame rate of the display  
>> patch and cutting down on cpu/gpu load.
>> Problem is, it seems only some codecs work with this method.  
>> Animation which you use I guess compresses over time, because when  
>> I try to display particular frames I get digital artifacts. For  
>> Jpeg, motion-jpeg the colors aren't right when the film is loading  
>> back in. I had success with PNG codec, but the file sizes after the  
>> compression are quite large.
>> Any other codecs you could recommend that compress each frame  
>> separately and work with this method of creating "quasi" uyvy to be  
>> decoded as such on the GPU?
>> To get Photo-Jpeg/Motion-Jpeg to work I tried to map the planes  
>> differently, but couldn't get the colors to match. Anyone throw  
>> some light on this? Is there a particular website that might  
>> describe all the different standard quicktime codecs? Detailing  
>> which compress every frame separately and which compress over time?  
>> Jitter does better with the former.
>> cheers
>> (())_n
>> On Dec 6, 2007, at 4:31 AM, yair reshef wrote:
>>> thank you mr. Dekron, your pre uyvy method has gotten me a 768x768  
>>> movie run a lot nicer on an xp.
>>>
>>> btw. is there a method to force other post enviroments  
>>> (aftereffects, shake etc.) to export to uyvy.
>>> most of the post friends i asked looked at me funny.
>>>
>>>
>>>
>>> On Nov 23, 2007 11:31 AM, John Dekron <pr at thisserver.de> wrote:
>>> Hi Tyler,
>>> I don't know about the funny colors you see, but on the fps side, i
>>> understand that jit.qt.movie is doing the conversion from argb to  
>>> uyvy
>>> internally. on some bigger movies I get even better performance  
>>> when I
>>> transform the source movie to a half sized "quasi" uyvy movie
>>> beforehand. (see attahced patch). I only see some "trailing" on that
>>> kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2
>>>
>>> John.
>>>
>>> max v2;
>>> #N vpatcher 461 240 1347 803;
>>> #P window setfont "Sans Serif" 9.;
>>> #P window linecount 1;
>>> #P comment 638 116 100 196617 5. read new movie;
>>> #P comment 119 209 100 196617 4. stop;
>>> #P window linecount 2;
>>> #P comment 116 179 100 196617 2. select write location;
>>> #P window linecount 1;
>>> #P message 597 119 30 196617 read;
>>> #P newex 554 193 297 196617 jit.gl.videoplane content @colormode  
>>> uyvy
>>> @transform_reset 2;
>>> #P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
>>> #P newex 490 307 94 196617 jit.window content;
>>> #P user jit.fpsgui 435 125 60 196617 0;
>>> #P newex 417 348 101 196617 jit.gl.render content;
>>> #P newex 417 97 66 196617 t b b b erase;
>>> #P toggle 417 48 15 0;
>>> #P newex 417 73 57 196617 qmetro 20;
>>> #P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
>>> #P message 90 197 29 196617 stop;
>>> #P message 91 160 154 196617 write 25. animation normal 600;
>>> #P newex 57 233 66 196617 jit.qt.record;
>>> #P message 122 66 30 196617 read;
>>> #P message 57 66 60 196617 framedump;
>>> #P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
>>> #P comment 125 45 100 196617 1. read movie;
>>> #P comment 21 45 100 196617 3. framedump;
>>> #P fasten 4 0 2 0 127 99 62 99;
>>> #P connect 3 0 2 0;
>>> #P connect 2 0 5 0;
>>> #P fasten 7 0 5 0 95 224 62 224;
>>> #P fasten 6 0 5 0 96 181 62 181;
>>> #P fasten 2 0 8 0 62 147 256 147;
>>> #P connect 10 0 9 0;
>>> #P connect 9 0 11 0;
>>> #P connect 11 0 12 0;
>>> #P fasten 11 3 12 0 476 160 422 160;
>>> #P connect 11 1 13 0;
>>> #P connect 17 0 15 0;
>>> #P fasten 11 2 15 0 458 121 559 121;
>>> #P connect 15 0 16 0;
>>> #P pop;
>>>
>>>
>>> Tyler Nitsch schrieb:
>>> > Upon vades suggestion and searching the archives....
>>> >
>>> > http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246
>>> >
>>> > I tried implementing this conversion on the gpu.  The problem is  
>>> I get funny colored purpley/greeny/red distorted lines on the  
>>> output window... and when I try to use either dx | qt.grab the  
>>> screen is distorted.
>>> >
>>> > The good news is my fps is WAY BETTER!!! without using  
>>> @colormode uyvy on the qt.movie I can only get 100fps on the patch  
>>> I'm posting whereas with the shader doing the conversion I get up  
>>> to 160 fps !!???!!  So weird.  Any help would greatly be  
>>> appreciated.
>>> >
>>> > I'm using windows XP PRO on an AMD s939 4800X2 with the latest  
>>> max and jitter.and a Viper ATI Radeon HD 3870 gpu.
>>> >
>>> > #P window setfont "Sans Serif" 9.;
>>> > #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0  
>>> 0 0;
>>> > #P window linecount 1;
>>> > #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1  
>>> @colormode argb;
>>> > #P newex 403 228 77 9109513 jit.gl.slab window;
>>> > #P user jit.fpsgui 155 341 60 9109513 0;
>>> > #P message 621 156 30 9109513 close;
>>> > #P newex 623 121 57 9109513 select 1 2 3;
>>> > #P message 652 156 28 9109513 open;
>>> > #P newex 388 154 42 9109513 gate 3 1;
>>> > #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0  
>>> @unique 1 @colormode uyvy;
>>> > #P message 80 398 62 9109513 fullscreen \$1;
>>> > #P toggle 80 380 15 0;
>>> > #P newex 58 330 40 9109513 key;
>>> > #P newex 80 357 46 9109513 select 27;
>>> > #P newex 141 227 250 9109513 jit.gl.slab window @file  
>>> cc.uyvy2rgba.jxs @dimscale 2. 1.;
>>> > #P newex 308 30 48 9109513 loadbang;
>>> > #P newex 139 84 30 9109513 t b b;
>>> > #P toggle 138 40 15 0;
>>> > #P newex 138 64 45 9109513 qmetro 5;
>>> > #P newex 75 121 111 9109513 t b b erase;
>>> > #P newex 193 308 192 9109513 jit.gl.videoplane window  
>>> @transform_reset 2;
>>> > #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1  
>>> @colormode uyvy;
>>> > #P message 310 117 76 9109513 read bball.mov;
>>> > #P newex 69 426 199 9109513 jit.window window @depthbuffer 1  
>>> @floating 1;
>>> > #P newex 75 299 86 9109513 jit.gl.render window;
>>> > #P connect 18 2 17 0;
>>> > #P connect 16 2 15 0;
>>> > #P connect 19 0 15 0;
>>> > #P connect 17 0 15 0;
>>> > #P connect 23 0 16 0;
>>> > #P connect 23 0 18 0;
>>> > #P connect 18 1 19 0;
>>> > #P connect 18 0 19 0;
>>> > #P connect 5 1 16 1;
>>> > #P connect 22 0 21 0;
>>> > #P connect 16 1 22 0;
>>> > #P fasten 2 0 3 0 315 144 147 144;
>>> > #P connect 2 0 22 0;
>>> > #P connect 9 0 2 0;
>>> > #P connect 10 0 4 0;
>>> > #P connect 21 0 4 0;
>>> > #P connect 0 0 20 0;
>>> > #P connect 16 0 3 0;
>>> > #P fasten 3 0 10 0 147 218 146 218;
>>> > #P fasten 15 0 10 0 628 218 146 218;
>>> > #P connect 6 0 8 0;
>>> > #P connect 7 0 6 0;
>>> > #P connect 13 0 14 0;
>>> > #P connect 11 0 13 0;
>>> > #P connect 12 0 11 0;
>>> > #P connect 5 0 0 0;
>>> > #P connect 5 2 0 0;
>>> > #P fasten 8 0 5 0 144 108 80 108;
>>> > #P connect 14 0 1 0;
>>> > #P window clipboard copycount 24;
>>> >
>>> >
>>> > _______________________________________________
>>> > jitter mailing list
>>> > jitter at cycling74.com
>>> > http://www.cycling74.com/mailman/listinfo/jitter
>>> >
>>>
>>> _______________________________________________
>>> jitter mailing list
>>> jitter at cycling74.com
>>> http://www.cycling74.com/mailman/listinfo/jitter
>>>
>>> _______________________________________________
>>> jitter mailing list
>>> jitter at cycling74.com
>>> http://www.cycling74.com/mailman/listinfo/jitter
>> _______________________________________________
>> jitter mailing list
>> jitter at cycling74.com
>> http://www.cycling74.com/mailman/listinfo/jitter
>
> _______________________________________________
> jitter mailing list
> jitter at cycling74.com
> http://www.cycling74.com/mailman/listinfo/jitter