Bytea on Windows JPEG problem with V2 BUG

[Jason Moehlman]

Hello,

I am experiencing a problem with storing bytea data only on Windows.
An older app that uses a canvas to accept a JPEG image and store it
in a bytea column no longer returns the same data it is submitted on
Windows.

I have traced it with the debugger and the proper data stream is sent
into the pgescapebytea routine, but when retrieved and unescaped the
first 2 words were "FFD8 FFC0" come back as "9FD8 9FC0" and are
rejected by the JPEG routine. I am not sure about the 2nd half of
the words, but I am sure the FF->9F is accurate, I did not write it
down.

The weirdest part is this only happens on WIndows, not Mac OS X or
Linux, those platforms both perform as expected and display the image
properly.

I have also verified that the same code with older v1 pgSQL4RB does
not have this problem and works properly on Windows also.

Anyone else experience this?

This is a critical bug for me and prevents shipping products with v2.

Jason

[Jason Moehlman]

Hello,

Forgot to mention that it will properly display on Windows if the
image is stored with a Mac or Linux version. It seems to be a
problem with Windows version of pgescapebytea for storing the
original image/data.

Jason

Post by Jason Moehlman
Hello,
I am experiencing a problem with storing bytea data only on Windows.
An older app that uses a canvas to accept a JPEG image and store it
in a bytea column no longer returns the same data it is submitted on
Windows.
I have traced it with the debugger and the proper data stream is sent
into the pgescapebytea routine, but when retrieved and unescaped the
first 2 words were "FFD8 FFC0" come back as "9FD8 9FC0" and are
rejected by the JPEG routine. I am not sure about the 2nd half of
the words, but I am sure the FF->9F is accurate, I did not write it
down.
The weirdest part is this only happens on WIndows, not Mac OS X or
Linux, those platforms both perform as expected and display the image
properly.
I have also verified that the same code with older v1 pgSQL4RB does
not have this problem and works properly on Windows also.
Anyone else experience this?
This is a critical bug for me and prevents shipping products with v2.
Jason
_______________________________________________
Postgresql mailing list
http://aliacta.com/mailman/listinfo/postgresql_aliacta.com

[Jason Moehlman]

Replacing fc3 or fc4 with fc2 eliminates the problem on Windows.

Jason

Post by Jason Moehlman
Hello,
Forgot to mention that it will properly display on Windows if the
image is stored with a Mac or Linux version. It seems to be a
problem with Windows version of pgescapebytea for storing the
original image/data.
Jason

Post by Jason Moehlman
Hello,
I am experiencing a problem with storing bytea data only on Windows.
An older app that uses a canvas to accept a JPEG image and store it
in a bytea column no longer returns the same data it is submitted on
Windows.
I have traced it with the debugger and the proper data stream is sent
into the pgescapebytea routine, but when retrieved and unescaped the
first 2 words were "FFD8 FFC0" come back as "9FD8 9FC0" and are
rejected by the JPEG routine. I am not sure about the 2nd half of
the words, but I am sure the FF->9F is accurate, I did not write it
down.
The weirdest part is this only happens on WIndows, not Mac OS X or
Linux, those platforms both perform as expected and display the image
properly.
I have also verified that the same code with older v1 pgSQL4RB does
not have this problem and works properly on Windows also.
Anyone else experience this?
This is a critical bug for me and prevents shipping products with v2.
Jason
_______________________________________________
Postgresql mailing list
http://aliacta.com/mailman/listinfo/postgresql_aliacta.com

_______________________________________________
Postgresql mailing list
http://aliacta.com/mailman/listinfo/postgresql_aliacta.com

[Aliacta Support]

Post by Jason Moehlman
Replacing fc3 or fc4 with fc2 eliminates the problem on Windows.

Thanks Jason, I'm going to have to look into this. Are all your
clients storing into the same server or do you have different
PostgreSQL databases around as well?

Marc

[Aliacta Support]

I figure everything works fine when you're replacing that code with
the extended query protocol though, isn't it? --Marc

[Jason Moehlman]

Marc,

I was going to look into that today. The document manager that I am
using the extended routines with was older and may not have the fc4
code in it at all. During routine testing I have just been adding
PDF files, extracting them and trying to open them. So far I have
not noticed anything, but I don't know how sensitive they are to the
first couple of bytes changing.

Either way I will be testing with a bit for bit verify and follow up
later today.

Jason

Post by Aliacta Support
I figure everything works fine when you're replacing that code with
the extended query protocol though, isn't it? --Marc
_______________________________________________
Postgresql mailing list
http://aliacta.com/mailman/listinfo/postgresql_aliacta.com

[Jason Moehlman]

Marc,

I have seen it on 7.4.X and 8.1.X so far. If you would like I can
send screen shots offlist from the debugger showing the exact byte
sequences before and after escaping, with the original image file, it
is fairly small.

I have not looked at whether or not the entire file is corrupted or
only the start. I will look into that today. The rest looked
similar enough, and the header was what was killing the JPEG image.

During the tracking, I also saw a couple of "Frontend error 243 or
234 in the PG console log. I am not sure what program generated them
or what version ( Win, Linux, etc. ) I don't know if that would even
help you at all. If I see it again today I will follow up.

If there is anything else I can do to help expedite please let me know.

Thanks again!

Jason

Post by Aliacta Support

Post by Jason Moehlman
Replacing fc3 or fc4 with fc2 eliminates the problem on Windows.

Thanks Jason, I'm going to have to look into this. Are all your
clients storing into the same server or do you have different
PostgreSQL databases around as well?
Marc
_______________________________________________
Postgresql mailing list
http://aliacta.com/mailman/listinfo/postgresql_aliacta.com

[Jason Moehlman]

Marc,

From what I can tell so far it is corrupting the whole file, only
where the upper nibble is a F and then not all. I spent some time in
a hex editor with the jpeg file and these are some of the "conversions"

FF -> 9F
F7 -> F7
F1 -> D1
F6 -> D6
F8 -> D8
F4 -> D4
FD -> DD
9E -> 8E
9A -> 8A
FE -> DE
FB -> DB
9C -> 8C

This is not a complete list, but you get the picture, so far I have
only found 9(x) and F(x) that are affected, except every F7 I found
is left alone.

This is also not an issue from what I can tell using the extended
query for the upload process, only using pgescapebytea on windows
with fc3 or fc4. I have uploaded the same file using fc4 with
extended mode query on windows and gotten back a valid file in
windows and mac using the standard pgunescapebytea routines with
simple queries.

I do not yet have a working "download" with extended query, but at
least I am getting valid pg-errors and not crashing with the
addition of the appenddescribe.

Hope this helps.

Jason

Post by Aliacta Support

Post by Jason Moehlman
Replacing fc3 or fc4 with fc2 eliminates the problem on Windows.

Thanks Jason, I'm going to have to look into this. Are all your
clients storing into the same server or do you have different
PostgreSQL databases around as well?
Marc
_______________________________________________
Postgresql mailing list
http://aliacta.com/mailman/listinfo/postgresql_aliacta.com