[jitter] cc.uyvy2rgba.jxs and optimization

HI Vade,

vade schrieb:
> Hm. Unless I am mistaken, UYVY does absolutely nothing with the alpha 
> channel what so ever. That information is lost to the ether unless you 
> use the alphaglue shader and read in the alpha via other means. UYVY has 
> no alpha channel, and I dont think that jit.qt.movie premulitplies alpha 
> for you while doing the conversion?
That is right. I was referring to the method where I store an uyvy movie 
in the argb animation codec. the alpha channel stores the u component.

What is the "official" codec for uyvy movies?

John.
> 
>  From my understanding of the UYVY shaders and colormode is:
> 
> *Some video codecs use an internal format that is something like ARGB 
> (or RGBA...), or YUV or some variant (Y, cr' cb' or whatnot)
> 
> *jit.qt.movie when left alone, will always do conversion to JITTERS 
> basic ARGB matrix format.
> 
> This means that any video whose codecs native internal pixel format is 
> YUV or some other needs to be converted to ARGB by either quicktime or 
> ji.qt.movie (I am unsure which internal mechanism does this). If the 
> pixel format is something like RGBA or ARGB, nothing really needs to be 
> done.
> 
> jit.qt.movie @colormode uyvy does the reverse of this, which means any 
> quicktime movie whose normal internal pixel format is YUV (or similarly 
> compatible) gets left alone, otherwise it is converted to UYVY.
> 
> This means that for codecs which normally use YUV or whatnot colorspace 
> internally, using @colormode UYVY is a fastpath - as there is no 
> conversion overhead.
> 
> This also means that ARGB codecs (or anything else), using @colormode 
> UYVY is NOT a fastpath, as conversion must happen (somewhere - again I 
> do not know specifically where this happens).
> 
> Id love to know from Cycling if this is correct.
> 
> *feature request below!*
> 
> Ive been thinking about this myself, and it might be nice if there was 
> an attribute @colorspace native
> 
> which would also dumpout the colorspace on the rightmost outlet (dumpout 
> outlet) right before  the movie plays its first frame, and load the 
> movie natively.
> 
> This would allow you to :
> 
> Have a fastpath GPU uploading solution that would retain alpha channel 
> should your quicktime movie have one, or skip right to UYVY those that 
> need it.
> 
> decide what to do with the alpha channel should you want to keep it.
> 
> 
> * As far as the errors go, have you tried the other various uyvy 
> conversion shaders, the lite, and the one with gamma correction?
> 
> As far as animation is concerned I belive it is
> 
> ARGB and uses only run length encoding, which is basically like 
> uncompressed, and has not inter frame coding. Its like motion jpeg/photo 
> jpeg, just ARGB and lossless compression.
> 
> 
> On Jan 2, 2008, at 7:09 AM, John Dekron wrote:
> 
>> Hi _n,
>> I get the digital artifacts as well and have no idea where they come 
>> from.
>> Animation and png are videocodecs that hold an alpha channel what is 
>> necessary for the uyvy information. If I remember correctly it is 
>> possible to limit the data rate on png compression. I did use png for 
>> a previous project, but lately I experienced some crashes when playing 
>> back png compressed movies. i switched to animation and it worked 
>> fine. Sorry that I didn't investigate deeper on the crashes that I 
>> had, so this information might as well be treated as gossip- but 
>> eventually there is some truth in it.
>>
>> good luck. John.
>>
>> (())_n schrieb:
>>> yes dekron, yair,
>>> I have been looking into this method of doing the conversion before, 
>>> and it does help, doubling the frame rate of the display patch and 
>>> cutting down on cpu/gpu load.
>>> Problem is, it seems only some codecs work with this method. 
>>> Animation which you use I guess compresses over time, because when I 
>>> try to display particular frames I get digital artifacts. For Jpeg, 
>>> motion-jpeg the colors aren't right when the film is loading back in. 
>>> I had success with PNG codec, but the file sizes after the 
>>> compression are quite large.
>>> Any other codecs you could recommend that compress each frame 
>>> separately and work with this method of creating "quasi" uyvy to be 
>>> decoded as such on the GPU?
>>> To get Photo-Jpeg/Motion-Jpeg to work I tried to map the planes 
>>> differently, but couldn't get the colors to match. Anyone throw some 
>>> light on this? Is there a particular website that might describe all 
>>> the different standard quicktime codecs? Detailing which compress 
>>> every frame separately and which compress over time? Jitter does 
>>> better with the former.
>>> cheers
>>> (())_n
>>> On Dec 6, 2007, at 4:31 AM, yair reshef wrote:
>>>> thank you mr. Dekron, your pre uyvy method has gotten me a 768x768 
>>>> movie run a lot nicer on an xp.
>>>>
>>>> btw. is there a method to force other post enviroments 
>>>> (aftereffects, shake etc.) to export to uyvy.
>>>> most of the post friends i asked looked at me funny.
>>>>
>>>>
>>>>
>>>> On Nov 23, 2007 11:31 AM, John Dekron <pr at thisserver.de> wrote:
>>>> Hi Tyler,
>>>> I don't know about the funny colors you see, but on the fps side, i
>>>> understand that jit.qt.movie is doing the conversion from argb to uyvy
>>>> internally. on some bigger movies I get even better performance when I
>>>> transform the source movie to a half sized "quasi" uyvy movie
>>>> beforehand. (see attahced patch). I only see some "trailing" on that
>>>> kind of material. this is on MacBook Pro, OSX 10.4.10, Quicktime 7.2
>>>>
>>>> John.
>>>>
>>>> max v2;
>>>> #N vpatcher 461 240 1347 803;
>>>> #P window setfont "Sans Serif" 9.;
>>>> #P window linecount 1;
>>>> #P comment 638 116 100 196617 5. read new movie;
>>>> #P comment 119 209 100 196617 4. stop;
>>>> #P window linecount 2;
>>>> #P comment 116 179 100 196617 2. select write location;
>>>> #P window linecount 1;
>>>> #P message 597 119 30 196617 read;
>>>> #P newex 554 193 297 196617 jit.gl.videoplane content @colormode uyvy
>>>> @transform_reset 2;
>>>> #P newex 554 167 155 196617 jit.qt.movie @adapt 1 @unique 1;
>>>> #P newex 490 307 94 196617 jit.window content;
>>>> #P user jit.fpsgui 435 125 60 196617 0;
>>>> #P newex 417 348 101 196617 jit.gl.render content;
>>>> #P newex 417 97 66 196617 t b b b erase;
>>>> #P toggle 417 48 15 0;
>>>> #P newex 417 73 57 196617 qmetro 20;
>>>> #P user jit.pwindow 250 161 82 62 0 1 0 0 1 0;
>>>> #P message 90 197 29 196617 stop;
>>>> #P message 91 160 154 196617 write 25. animation normal 600;
>>>> #P newex 57 233 66 196617 jit.qt.record;
>>>> #P message 122 66 30 196617 read;
>>>> #P message 57 66 60 196617 framedump;
>>>> #P newex 57 123 190 196617 jit.qt.movie @adapt 1 @colormode uyvy;
>>>> #P comment 125 45 100 196617 1. read movie;
>>>> #P comment 21 45 100 196617 3. framedump;
>>>> #P fasten 4 0 2 0 127 99 62 99;
>>>> #P connect 3 0 2 0;
>>>> #P connect 2 0 5 0;
>>>> #P fasten 7 0 5 0 95 224 62 224;
>>>> #P fasten 6 0 5 0 96 181 62 181;
>>>> #P fasten 2 0 8 0 62 147 256 147;
>>>> #P connect 10 0 9 0;
>>>> #P connect 9 0 11 0;
>>>> #P connect 11 0 12 0;
>>>> #P fasten 11 3 12 0 476 160 422 160;
>>>> #P connect 11 1 13 0;
>>>> #P connect 17 0 15 0;
>>>> #P fasten 11 2 15 0 458 121 559 121;
>>>> #P connect 15 0 16 0;
>>>> #P pop;
>>>>
>>>>
>>>> Tyler Nitsch schrieb:
>>>> > Upon vades suggestion and searching the archives....
>>>> >
>>>> > 
>>>> http://www.cycling74.com/forums/index.php?t=msg&goto=113246&rid=0&S=12672e78d4a9aad871b2693f9a7bbb5b#msg_113246 
>>>>
>>>> >
>>>> > I tried implementing this conversion on the gpu.  The problem is I 
>>>> get funny colored purpley/greeny/red distorted lines on the output 
>>>> window... and when I try to use either dx | qt.grab the screen is 
>>>> distorted.
>>>> >
>>>> > The good news is my fps is WAY BETTER!!! without using @colormode 
>>>> uyvy on the qt.movie I can only get 100fps on the patch I'm posting 
>>>> whereas with the shader doing the conversion I get up to 160 fps 
>>>> !!???!!  So weird.  Any help would greatly be appreciated.
>>>> >
>>>> > I'm using windows XP PRO on an AMD s939 4800X2 with the latest max 
>>>> and jitter.and a Viper ATI Radeon HD 3870 gpu.
>>>> >
>>>> > #P window setfont "Sans Serif" 9.;
>>>> > #P number 418 72 35 9 1 3 3 139 0 0 0 221 221 221 222 222 222 0 0 0;
>>>> > #P window linecount 1;
>>>> > #P newex 403 193 214 9109513 jit.qt.movie 720 480 @unique 1 
>>>> @colormode argb;
>>>> > #P newex 403 228 77 9109513 jit.gl.slab window;
>>>> > #P user jit.fpsgui 155 341 60 9109513 0;
>>>> > #P message 621 156 30 9109513 close;
>>>> > #P newex 623 121 57 9109513 select 1 2 3;
>>>> > #P message 652 156 28 9109513 open;
>>>> > #P newex 388 154 42 9109513 gate 3 1;
>>>> > #P newex 623 193 267 9109513 jit.dx.grab 720 480 @vdevice 0 
>>>> @unique 1 @colormode uyvy;
>>>> > #P message 80 398 62 9109513 fullscreen \$1;
>>>> > #P toggle 80 380 15 0;
>>>> > #P newex 58 330 40 9109513 key;
>>>> > #P newex 80 357 46 9109513 select 27;
>>>> > #P newex 141 227 250 9109513 jit.gl.slab window @file 
>>>> cc.uyvy2rgba.jxs @dimscale 2. 1.;
>>>> > #P newex 308 30 48 9109513 loadbang;
>>>> > #P newex 139 84 30 9109513 t b b;
>>>> > #P toggle 138 40 15 0;
>>>> > #P newex 138 64 45 9109513 qmetro 5;
>>>> > #P newex 75 121 111 9109513 t b b erase;
>>>> > #P newex 193 308 192 9109513 jit.gl.videoplane window 
>>>> @transform_reset 2;
>>>> > #P newex 142 188 219 9109513 jit.qt.movie 720 480 @unique 1 
>>>> @colormode uyvy;
>>>> > #P message 310 117 76 9109513 read bball.mov;
>>>> > #P newex 69 426 199 9109513 jit.window window @depthbuffer 1 
>>>> @floating 1;
>>>> > #P newex 75 299 86 9109513 jit.gl.render window;
>>>> > #P connect 18 2 17 0;
>>>> > #P connect 16 2 15 0;
>>>> > #P connect 19 0 15 0;
>>>> > #P connect 17 0 15 0;
>>>> > #P connect 23 0 16 0;
>>>> > #P connect 23 0 18 0;
>>>> > #P connect 18 1 19 0;
>>>> > #P connect 18 0 19 0;
>>>> > #P connect 5 1 16 1;
>>>> > #P connect 22 0 21 0;
>>>> > #P connect 16 1 22 0;
>>>> > #P fasten 2 0 3 0 315 144 147 144;
>>>> > #P connect 2 0 22 0;
>>>> > #P connect 9 0 2 0;
>>>> > #P connect 10 0 4 0;
>>>> > #P connect 21 0 4 0;
>>>> > #P connect 0 0 20 0;
>>>> > #P connect 16 0 3 0;
>>>> > #P fasten 3 0 10 0 147 218 146 218;
>>>> > #P fasten 15 0 10 0 628 218 146 218;
>>>> > #P connect 6 0 8 0;
>>>> > #P connect 7 0 6 0;
>>>> > #P connect 13 0 14 0;
>>>> > #P connect 11 0 13 0;
>>>> > #P connect 12 0 11 0;
>>>> > #P connect 5 0 0 0;
>>>> > #P connect 5 2 0 0;
>>>> > #P fasten 8 0 5 0 144 108 80 108;
>>>> > #P connect 14 0 1 0;
>>>> > #P window clipboard copycount 24;
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > jitter mailing list
>>>> > jitter at cycling74.com
>>>> > http://www.cycling74.com/mailman/listinfo/jitter
>>>> >
>>>>
>>>> _______________________________________________
>>>> jitter mailing list
>>>> jitter at cycling74.com
>>>> http://www.cycling74.com/mailman/listinfo/jitter
>>>>
>>>> _______________________________________________
>>>> jitter mailing list
>>>> jitter at cycling74.com
>>>> http://www.cycling74.com/mailman/listinfo/jitter
>>> _______________________________________________
>>> jitter mailing list
>>> jitter at cycling74.com
>>> http://www.cycling74.com/mailman/listinfo/jitter
>>
>> _______________________________________________
>> jitter mailing list
>> jitter at cycling74.com
>> http://www.cycling74.com/mailman/listinfo/jitter
> 
> _______________________________________________
> jitter mailing list
> jitter at cycling74.com
> http://www.cycling74.com/mailman/listinfo/jitter
>