Crash in Art-s dbcopy

Crash in Art-s dbcopy

Post by Dirk B » Tue, 19 Aug 2008 01:50:22


I-m migration a database from Solaris 2.8 9.40FC5 to Suse 11.50FC1 using
Art's dbcopy program. It crashes on every table that has a
lvarchar-column with "code -1831". I can-t even find this error-code ...
All other tables work fine.
Has anyone successfully transferred tables with lvarchars and dbcopy?
Or an idea what might causing the crash?

Thanks

Dirk
 
 
 

Crash in Art-s dbcopy

Post by Art Kage » Tue, 19 Aug 2008 04:30:36

t would seem to be a bug in the ESQL/C library or IDS itself. I have just
finished extensive testing. Dbcopy works fine for LVARCHAR columns as long
as the length of the column is 6,546 bytes or less. I was sure that I'd
tested LVARCHARs when I added support for them 8-). I will open a case with
IBM on Monday to track this. Thanks for reporting it Dirk.

Note from 6547 bytes to about 10,000 bytes dbschema gets the undocumented
error -1831. From about 10,000 bytes and up it does SEGV while fetching the
first row, but that may just be a more severe symptom of the same bug.

Art

On Sun, Aug 17, 2008 at 12:50 PM, Dirk B. < XXXX@XXXXX.COM > wrote:




--
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors ( XXXX@XXXXX.COM )

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do those
opinions reflect those of other individuals affiliated with any entity with
which I am affiliated nor those of the entities themselves.

<div dir="ltr">It would seem to be a bug in the ESQL/C library or IDS itself.  I have just finished extensive testing.  Dbcopy works fine for LVARCHAR columns as long as the length of the column is 6,546 bytes or less.  I was sure that I'd tested LVARCHARs when I added support for them 8-).  I will open a case with IBM on Monday to track this.  Thanks for reporting it Dirk.<br>
<br>Note from 6547 bytes to about 10,000 bytes dbschema gets the undocumented error -1831.  From about 10,000 bytes and up it does SEGV while fetching the first row, but that may just be a more severe symptom of the same bug.<br>
<br>Art<br><br><div class="gmail_quote">On Sun, Aug 17, 2008 at 12:50 PM, Dirk B. <span dir="ltr"><<a href="mailto: XXXX@XXXXX.COM "> XXXX@XXXXX.COM </a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I-m migration a database from Solaris 2.8 9.40FC5 to Suse 11.50FC1 using<br>
Art's dbcopy program. It crashes on every table that has a<br>
lvarchar-column with "code -1831". I can-t even find this error-code ...<br>
All other tables work fine.<br>
Has anyone successfully transferred tables with lvarchars and dbcopy?<br>
Or an idea what might causing the crash?<br>
<br>
Thanks<br>
<br>
Dirk<br>
_______________________________________________<br>
Informix-list mailing list<br>
<a href="mailto: XXXX@XXXXX.COM "> XXXX@XXXXX.COM </a><br>
<a href="http://www.iiug.org/mailman/listinfo/informix-list" target="_blank">http://www.iiug.org/mailman/listinfo/informix-list</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>Art S. Kagel<br>Oninit (<a href="http://www.oninit.com">www.oninit.com</a>)<br>IIUG Board of Directors (<a href="mailto: XXXX@XXXXX.COM "> XXXX@XXXXX.COM </a>)<br><br>Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Oninit, the IIUG, nor any other organization with which I am associated either explicitly o
 
 
 

Crash in Art-s dbcopy

Post by Jonathan L » Tue, 19 Aug 2008 07:21:11

n Sun, Aug 17, 2008 at 12:30 PM, Art Kagel < XXXX@XXXXX.COM > wrote:

FWIW, I have:

Black JL: chkmsg -1831
-1831: Combination of FetArrSize Deferred-PREPARE and OPTOFC is not supported.
Black JL:

I'm not sure what that has to do with the problem...

Art - testing with CSDK 3.50.FC1 and IDS 11.50.FC1 on Solaris 10 (and
SQLCMD 86.00), I used this test script:

sqlcmd -C -d stores 'create table lvc(lvc lvarchar(20000) not null, s
serial not null primary key)'

perl -e '
my(@list) = (1, 10, 100, 1000, 6000, 6300, 6400, 6500, 6600,
6700, 6800, 6900, 7000, 8000, 9000, 10000, 11000,
12000, 19000, 19999, 20000, 20001);
foreach my $i (@list)
{
print "X" x $i, "|0|\n";
}' |
sqlcmd -R -d stores -t lvc -N 1
sqlcmd -U -d stores -t lvc -O s |
awk -F'|' '{print length($1);}'

The output from that script was:


Black JL: ksh -x test.lvc.sh
+ sqlcmd -C -d stores create table lvc(lvc lvarchar(20000) not null, s
serial not null primary key)
+ perl -e
my(@list) = (1, 10, 100, 1000, 6000, 6300, 6400, 6500, 6600,
6700, 6800, 6900, 7000, 8000, 9000, 10000, 11000,
12000, 19000, 19999, 20000, 20001);
foreach my $i (@list)
{
print "X" x $i, "|0|\n";
}
+ sqlcmd -R -d stores -t lvc -N 1
1 rows committed
2 rows committed
3 rows committed
4 rows committed
5 rows committed
6 rows committed
7 rows committed
8 rows committed
9 rows committed
10 rows committed
11 rows committed
12 rows committed
13 rows committed
14 rows committed
15 rows committed
16 rows committed
17 rows committed
18 rows committed
19 rows committed
20 rows committed
21 rows committed
22 rows committed
+ sqlcmd -U -d stores -t lvc -O s
+ awk -F| {print length($1);}
1
10
100
1000
6000
6300
6400
6500
6600
6700
6800
6900
7000
8000
9000
10000
11000
12000
19000
19999
20000
20000
Black JL:

While this is by no means conclusive proof that there is not a problem
on some platforms, or with some versions of IDS, it does suggest that
there are some platforms where LVARCHAR(20000) works OK.





--
Jonathan Leffler #include <disclaimer.h>
Email: XXXX@XXXXX.COM , XXXX@XXXXX.COM
Guardian of DBD::Informix v2008.0513 -- http://dbi.perl.org/
"Blessed are we who can laugh at ourselves, for we shall never cease
to be amused."
NB: Please do not use this email for correspondence.
I don't necessarily read it every week, even.
 
 
 

Crash in Art-s dbcopy

Post by Art Kage » Wed, 20 Aug 2008 02:47:56

xcept dbcopy doesn't use deferred PREPARE or OPTOFC. It does use
FetArrSize though.

The very odd thing is that with the column sized at 6546 with the SERIAL
column I also included in the test, the rowsize is 6550.

A maximum FetBufSize buffer of 32767 bytes will hold 5 such rows, but it
will also hold 5 rows of 6551 bytes....

Hmm, maybe with the column alignment ... Ya, assuming the alignment
adjustment to space out for the oddsized LVARCHAR caused dbcopy to bring the
required buffer size for the 6547 bytes column version up to 6554 and the
comm buffer will only hold 4.995 - ie four rows and I've set the FetArrBuf
for five rows. Might be the problem. I'll check it out. It's likely that
I've never tested with a rowsize that was sized such that the field
alignment caused the buffers to be too big.

Dirk, can you verify what your rowsize is for the dbcopy that's failing?

Art

On Sun, Aug 17, 2008 at 6:21 PM, Jonathan Leffler
< XXXX@XXXXX.COM >wrote:




--
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors ( XXXX@XXXXX.COM )

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do those
opinions reflect those of other individuals affiliated with any entity with
which I am affiliated nor those of the entities themselves.

<div dir="ltr">Except dbcopy doesn't use deferred PREPARE or OPTOFC.  It does use FetArrSize though.<br><br>The very odd thing is that with the column sized at 6546 with the SERIAL column I also included in the test, the rowsize is 6550.<br>
<br>A maximum FetBufSize buffer of 32767 bytes will hold 5 such rows, but it will also hold 5 rows of 6551 bytes....<br><br>Hmm, maybe with the column alignment ... Ya, assuming the alignment adjustment to space out for the oddsized LVARCHAR caused dbcopy to bring the required buffer size for the 6547 bytes column version up to 6554 and the comm buffer will only hold 4.995 - ie four rows and I've set the FetArrBuf for five rows.  Might be the problem.  I'll check it out.  It's likely that I've never tested with a rowsize that was sized such that the field alignment caused the buffers to be too big.<br>
<br>Dirk, can you verify what your rowsize is for the dbcopy that's failing?<br><br>Art<br><br><div class="gmail_quote">On Sun, Aug 17, 2008 at 6:21 PM, Jonathan Leffler <span dir="ltr"><<a href="mailto: XXXX@XXXXX.COM "> XXXX@XXXXX.COM </a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div class="Ih2E3d">On Sun, Aug 17, 2008 at 12:30 PM, Art Kagel <<a href="mailto: XXXX@XXXXX.COM "> XXXX@XXXXX.COM </a>> wrote:<br>

<br>
</div>FWIW, I have:<br>
<br>
Black JL: chkmsg -1831<br>
-1831: Combination of FetArrSize Deferred-PREPARE and OPTOFC is not supported.<br>
Black JL:<br>
<br>
I'm not sure what that has to do with the problem...<br>
<br>
Art - testing with CSDK 3.50.FC1 and IDS 11.50.FC1 on Solaris 10 (and<br>
SQLCMD 86.00), I used this test script:<br>
<br>
sqlcmd -C
 
 
 

Crash in Art-s dbcopy

Post by Art Kage » Wed, 20 Aug 2008 02:56:22

o. I recoded dbcopy to add 3 bytes for alignment (simplistic but it is
correct for this one table) which caused it to calculate that only 4 rows
fit in a comm buffer. But it still gets SQLCODE = -1831

Art

On Mon, Aug 18, 2008 at 1:47 PM, Art Kagel < XXXX@XXXXX.COM > wrote:



--
Art S. Kagel
Oninit (www.oninit.com)
IIUG Board of Directors ( XXXX@XXXXX.COM )

Disclaimer: Please keep in mind that my own opinions are my own opinions and
do not reflect on my employer, Oninit, the IIUG, nor any other organization
with which I am associated either explicitly or implicitly. Neither do those
opinions reflect those of other individuals affiliated with any entity with
which I am affiliated nor those of the entities themselves.

<div dir="ltr">No.  I recoded dbcopy to add 3 bytes for alignment (simplistic but it is correct for this one table) which caused it to calculate that only 4 rows fit in a comm buffer.  But it still gets SQLCODE = -1831<br>
<br>Art<br><br><div class="gmail_quote">On Mon, Aug 18, 2008 at 1:47 PM, Art Kagel <span dir="ltr"><<a href="mailto: XXXX@XXXXX.COM "> XXXX@XXXXX.COM </a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div dir="ltr">Except dbcopy doesn't use deferred PREPARE or OPTOFC.  It does use FetArrSize though.<br><br>The very odd thing is that with the column sized at 6546 with the SERIAL column I also included in the test, the rowsize is 6550.<br>

<br>A maximum FetBufSize buffer of 32767 bytes will hold 5 such rows, but it will also hold 5 rows of 6551 bytes....<br><br>Hmm, maybe with the column alignment ... Ya, assuming the alignment adjustment to space out for the oddsized LVARCHAR caused dbcopy to bring the required buffer size for the 6547 bytes column version up to 6554 and the comm buffer will only hold 4.995 - ie four rows and I've set the FetArrBuf for five rows.  Might be the problem.  I'll check it out.  It's likely that I've never tested with a rowsize that was sized such that the field alignment caused the buffers to be too big.<br>

<br>Dirk, can you verify what your rowsize is for the dbcopy that's failing?<br><font color="#888888"><br>Art</font><div><div></div><div class="Wj3C7c"><br><br><div class="gmail_quote">On Sun, Aug 17, 2008 at 6:21 PM, Jonathan Leffler <span dir="ltr"><<a href="mailto: XXXX@XXXXX.COM " target="_blank"> XXXX@XXXXX.COM </a>></span> wrote:<br>

<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div>On Sun, Aug 17, 2008 at 12:30 PM, Art Kagel <<a href="mailto: XXXX@XXXXX.COM " target="_blank"> XXXX@XXXXX.COM </a>> wrote:<br>


<br>
</div>FWIW, I have:<br>
<br>
Black JL: chkmsg -1831<br>
-1831: Combination of FetArrSize Deferred-PREPARE and OPTOFC is not supported.<br>
Black JL:<br>
<br>
I'm not sure what that has to do with the problem...<br>
<br>
Art - testing with CSDK 3.50.FC1 and IDS 11.50.FC1 on Solaris 10 (and<br>
SQLCMD 86.00), I used this test script:
 
 
 

Crash in Art-s dbcopy

Post by Dirk B » Wed, 20 Aug 2008 19:17:59

Art, i don-t understand the rowsize calculation of dbcopy.
A simple table like this:

create table "informix".tab1
(
tb1_id serial not null ,
tb1_tb3_id integer not null
}

I would expect 8 as the rowsize but dbcopy says 16. But table is copied
without problems.

On this one with lvarchars:

create table "informix".tab2
(
tb2_id serial not null ,
tb2_tb4_id integer not null ,
tb2_line_no integer,
tb2_retcode smallint not null ,
tb2_line lvarchar not null ,
tb2_more_info lvarchar,
tb2_man_id integer not null ,
tb2_grp_eig integer not null ,
tb2_mar_crea integer not null ,
tb2_crea_dat datetime year to second not null ,
tb2_mar_upd integer not null ,
tb2_upd_dat datetime year to second not null
)

dbschema says 4148, dbcopy 4218. And this one crashes with -1831.

Or is the rowsize that dbcopy calculates the regular rowsize + something?




Dirk