OnDemand Users Group

Support Forums => Report Indexing => Topic started by: DDP021 on July 30, 2019, 12:29:41 PM

Title: Bad Character???
Post by: DDP021 on July 30, 2019, 12:29:41 PM
Hi, we have an old report defined using OS390 indexer..Its been running for years...recently its been failing on an index (USER NAME) with the below error:

The string "SISSOKO, A��SSATA " has a length of 20 and the field has a maximum length of 18

I know its do to either the character that looks like an O with a line through (x'80') it OR the I with the 2 dots above it (x'57') because when we remove those 2 characters, the file loads.  We did attempt to remove each one individually and received the same results..But removing both allowed the file to load successfully...We even attempted to created a new definition using ACIF indexer with same results...Had anyone else ran into this kind of issue with special characters? Also, the INDEX is set as a STRING.

Thanks in advance for any input

Take care

Dave
Title: Re: Bad Character???
Post by: Ed_Arnold on July 30, 2019, 02:15:20 PM
Hello Dave - I ran into this in 2016 on z. 

Here's some discussion I had on this:

Quote
With 8.5 comes greater accuracy with handling of non-US characters.
Typically, this is a symptom of the German "Sharp S" which when
uppercased becomes two letters.

Here's how you fix it:

a) install PM82660/UK92465

b) add the following to ars.cfg:
     ARS_USE_OLD_STYLE_CASE=1

That's all that's needed to resolve the loading problem.

However,

c) all clients should be at 8.5.0.6 or higher to deal with the
character being stored as a Sharp S when doing searches.
.....................................................................

Related APAR:

.....................................................................

APAR PM70770 / UK81470 Description

Upper-casing the German s-sharp produces SS.  This expansion of
the string caused the NULL that  terminated the string to be
overlaid, causing subsequent string operations to behave
incorrectly.

PROBLEM CONCLUSION:
****************************************************************
* PROBLEM DESCRIPTION: When running arsload with a applgrp     *
*                      field that specifies upper-casing and   *
*                      the field contains a German s-sharp     *
*                      (U+00DF), the field can either be       *
*                      truncated, or arsload can abend due to  *
*                      heap  corruption.                       *
****************************************************************
* RECOMMENDATION:                                              *
****************************************************************
ARNADMP is changed to detect the expansion of the string due to
the upper-case operation and not overlay the trailing NUL.
Additionally, ARNADMP will detect if the expansion exceeds the
size of the field and issue an error message.

..................................................................

This issue was reported on z and fixed on z.

But then along came the problem on AIX.

For indexing and loading on AIX you also add the following to ars.cfg:

     ARS_USE_OLD_STYLE_CASE=1




Just for completeness of this discussion, there was still a problem loading from AIX -> z.

A colleague weighed in with this:

Quote
I have been looking at how to fix this issue and trying to use ars.cfg /
ars.ini on your AIX machine may prove to be a bit tricky.

From my tests here on the AIX box you can also specify an environment
variable. Do this for the user(s) who run the arsload command:

ARS_USE_OLD_STYLE_CASE=1

The reply was

Quote
...it works!! ...

I put

export ARS_USE_OLD_STYLE_CASE=1

before the call of arsload

Ed
Title: Re: Bad Character???
Post by: DDP021 on July 30, 2019, 02:24:03 PM
Thanks Ed!...We are running Version 9.5.0.8

I will pass this info onto our systems group to see what they can possibly do.

Appreciate the info!!!!!

Take care

Dave
Title: Re: Bad Character???
Post by: Ed_Arnold on July 30, 2019, 02:28:59 PM
Dave - I updated my post after your last post.

Just FYI to pick up the (minimal) changes.

Hope this helps!

Ed
Title: Re: Bad Character???
Post by: Justin Derrick on July 30, 2019, 02:50:09 PM
This issue was also mentioned in one of the previous ODNews newsletters:  http://www.ibm.com/support/docview.wss?uid=swg27041378

-JD.