Encoding, Unicode and Lasso database broswer app

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Encoding, Unicode and Lasso database broswer app

Clive Bruton
I just know I'm doing something incredibly stupid here, but as usual,  
I've no idea what that might be.

Lasso 8.0.4, MySQL 4.1.12, MacOS X 10.4.2:

I've gone through all the tables in a database and set their encoding  
to UTF-8 (with MySQL Administrator), I've also gone through Site  
Admin > Data sources > Tables and set them all to UTF-8 there as well.

Then I go to the Lasso database browser and see what's in the  
database, and in a record view I see something like:

     product_desc:     <p>Issue eleven... </p>

Which is what I'd sort of expect because there is HTML code in the  
description. When I click the Update Record button the data is  
presented in a textarea like this:

     &lt;p&gt;Issue eleven... &lt;/p&gt;

Which is exactly what I don't want. I'm presuming that this is just  
some kind of error because I changed the encoding, but when I edit  
and update with:

     <p>Issue eleven... </p>

It just comes back the same:

     &lt;p&gt;Issue eleven... &lt;/p&gt;

If I don't change the angle brackets it gets worse:

     &amp;lt;p&amp;gt;Issue eleven...

And I can only assume that gets incrementally worse with every  
"update" click.

What am I doing wrong?


-- Clive

--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Pierre Yelle
Hello Clive,

if you add a new record will it show correctly ?

If so, the tables matadata has been flagged but the existing data was
not converted.

Maybe undo the MySQL Admin pref you did earlier and see if tables are
as before then
issue


ALTER TABLE table_name CONVERT TO CHARACTER SET utf8 COLLATE
utf8_unicode_ci

Maybe it is too late now to apply this

Other solution is a per column statement with full column specs + new
character definition

ALTER TABLE t1 CHANGE c1 c1 TEXT CHARACTER SET utf8 COLLATE
utf8_unicode_ci

Try these on test table first.

good luck

Pierre



On 14-Sep-05, at 4:13 PM, Clive Bruton wrote:

> I just know I'm doing something incredibly stupid here, but as usual,
> I've no idea what that might be.
>
> Lasso 8.0.4, MySQL 4.1.12, MacOS X 10.4.2:
>
> I've gone through all the tables in a database and set their encoding
> to UTF-8 (with MySQL Administrator), I've also gone through Site Admin
> > Data sources > Tables and set them all to UTF-8 there as well.
>
> Then I go to the Lasso database browser and see what's in the
> database, and in a record view I see something like:
>
>     product_desc:     <p>Issue eleven... </p>
>
> Which is what I'd sort of expect because there is HTML code in the
> description. When I click the Update Record button the data is
> presented in a textarea like this:
>
>     &lt;p&gt;Issue eleven... &lt;/p&gt;
>
> Which is exactly what I don't want. I'm presuming that this is just
> some kind of error because I changed the encoding, but when I edit and
> update with:
>
>     <p>Issue eleven... </p>
>
> It just comes back the same:
>
>     &lt;p&gt;Issue eleven... &lt;/p&gt;
>
> If I don't change the angle brackets it gets worse:
>
>     &amp;lt;p&amp;gt;Issue eleven...
>
> And I can only assume that gets incrementally worse with every
> "update" click.
>
> What am I doing wrong?
>
>
> -- Clive
>
> --
> ------------------------------
> Lasso Support: http://support.omnipilot.com/
> Search the list archives: http://www.listsearch.com/lassotalk.lasso
> Manage your list subscription:
> http://www.listsearch.com/lassotalk.lasso?manage
>


--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Clive Bruton
In reply to this post by Clive Bruton

On 15 Sep 2005, at 01:15, Pierre Yelle wrote:

> if you add a new record will it show correctly ?

Well, it's a bit curious, in that in the record view (ie when not  
editing) I can't see the <p> tags that I've added in, but when I  
click the update button to edit the string is encoded:

     &lt;p&gt;Hello world&lt;/p&gt;

Then it follows the same pattern of encoding again with each click of  
the update button:

     &amp;lt;p&amp;gt;Hello world&amp;lt;/p&amp;gt;

     &amp;amp;lt;p&amp;amp;gt;Hello world&amp;amp;lt;/p&amp;amp;gt;

> ALTER TABLE table_name CONVERT TO CHARACTER SET utf8 COLLATE  
> utf8_unicode_ci

But that's what the MySQL admin app did anyway, without the collate  
(because what you have is the default).

I've checked this with another database/table that's using ISO Latin  
1 encoding, and it seems that's doing the same thing (ie doubling-up  
the encoding).

> Maybe it is too late now to apply this

I think the text is actually ok, and has converted correctly, because  
it looks as it should if I view via CocoaMySQL. If I do queries  
through MySQL the data look alright too:

Entered in Lasso database browser:

     <p>Hello world – it’s ok</p>

Shown in MySQL:

     <p>Hello world – it’s ok</p> <!--in the terminal there's a  
space before the apostrophe!?-->

Returned in Lasso database browser (edit mode):

     &lt;p&gt;Hello world &ndash; it&rsquo;s ok&lt;/p&gt;

Source of the above:

     <textarea name="update_productDescription"  
class="browse_area_medium" wrap="virtual">
     &amp;lt;p&amp;gt;Hello world &amp;ndash; it&amp;rsquo;s  
ok&amp;lt;/p&amp;gt;
     </textarea>

Similarly, if I enter some (random) Greek:

     αεξμλν

In MySQL looks like:

     αεξμλν (but has spaces before each character, copies/pastes  
the same)

In Lasso database browser (edit mode):

     &alpha;&epsilon;&xi;&mu;&lambda;&nu;

In Lasso database browser (view mode):

     αεξμλν (ie ok)

So, er, it looks like the data is going into the database correctly,  
but double encoding on the way back (whether the table is Unicode or  
ISO Latin 1).


-- Clive
--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Pierre Yelle
In reply to this post by Clive Bruton

On 14-Sep-05, at 6:19 PM, Clive Bruton wrote:

>
>
>> ALTER TABLE table_name CONVERT TO CHARACTER SET utf8 COLLATE
>> utf8_unicode_ci
>
>


If you check the log, did the Admin Pref chage you operated actually
did the above statement (CONVERT)
.... or the following

ALTER TABLE tbl_name DEFAULT CHARACTER SET charset_name

... in which case the data remained unchanged


Pierre


--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Clive Bruton
In reply to this post by Clive Bruton

On 15 Sep 2005, at 03:09, Pierre Yelle wrote:

> If you check the log, did the Admin Pref chage you operated  
> actually did the above statement (CONVERT)
> .... or the following

The MySQL Administrator app says that the following statement was  
issued:

     ALTER TABLE `testshop`.`products` CHARACTER SET utf8;

Is that right?

Either way, I think "new" stuff should be ok, which in terms of raw  
characters is right - but there's still the double encoding going on  
in the Lasso database browser.


-- Clive

--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Pierre Yelle
In reply to this post by Clive Bruton

On 14-Sep-05, at 7:24 PM, Clive Bruton wrote:

>
> On 15 Sep 2005, at 03:09, Pierre Yelle wrote:
>
>> If you check the log, did the Admin Pref chage you operated actually
>> did the above statement (CONVERT)
>> .... or the following
>
> The MySQL Administrator app says that the following statement was
> issued:
>
>     ALTER TABLE `testshop`.`products` CHARACTER SET utf8;
>
> Is that right?

Yes, that should have changed the values to utf8...
>
> Either way, I think "new" stuff should be ok, which in terms of raw
> characters is right - but there's still the double encoding going on
> in the Lasso database browser.
>

Resart lasso is my next suggestion ??/



> -- Clive
>
> --
> ------------------------------
> Lasso Support: http://support.omnipilot.com/
> Search the list archives: http://www.listsearch.com/lassotalk.lasso
> Manage your list subscription:
> http://www.listsearch.com/lassotalk.lasso?manage
>


--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Clive Bruton
In reply to this post by Clive Bruton

On 15 Sep 2005, at 03:50, Pierre Yelle wrote:

> Resart lasso is my next suggestion ??/

Restarted the whole system, and upgraded to 8.0.5, no change.


-- Clive

--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Pierre Yelle
In reply to this post by Clive Bruton

On 14-Sep-05, at 8:17 PM, Clive Bruton wrote:

>
> On 15 Sep 2005, at 03:50, Pierre Yelle wrote:
>
>> Resart lasso is my next suggestion ??/
>
> Restarted the whole system, and upgraded to 8.0.5, no change.
>
>
>


First time I look at my data via the Lasso database browser and I get
the same error in edit mode.
My tables are all utf8.


Must be the Encoding error that will be resolbed in 8.06.
Greg Willits had a word on this in an earlier message today:

-----------
It's a bug in Lasso. The work around (found by someone) is to wrap it
in ().

     [(var:'yourvar', -encodebreak)]
-----------------




--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Adam Randall
In reply to this post by Clive Bruton
It's a simple bug in the DatabaseBrowser.LassoApp. If you want a fix,  
please join the beta program.

Adam.

--
Adam Randall <[hidden email]>
Quality Assurance / Support
OmniPilot Software, Inc.
http://www.omnipilot.com/
(954) 874-3100
AIM/iChat: adamomni


On Sep 14, 2005, at 7:13 PM, Clive Bruton wrote:

> I just know I'm doing something incredibly stupid here, but as  
> usual, I've no idea what that might be.
>
> Lasso 8.0.4, MySQL 4.1.12, MacOS X 10.4.2:
>
> I've gone through all the tables in a database and set their  
> encoding to UTF-8 (with MySQL Administrator), I've also gone  
> through Site Admin > Data sources > Tables and set them all to  
> UTF-8 there as well.
>
> Then I go to the Lasso database browser and see what's in the  
> database, and in a record view I see something like:
>
>     product_desc:     <p>Issue eleven... </p>
>
> Which is what I'd sort of expect because there is HTML code in the  
> description. When I click the Update Record button the data is  
> presented in a textarea like this:
>
>     &lt;p&gt;Issue eleven... &lt;/p&gt;
>
> Which is exactly what I don't want. I'm presuming that this is just  
> some kind of error because I changed the encoding, but when I edit  
> and update with:
>
>     <p>Issue eleven... </p>
>
> It just comes back the same:
>
>     &lt;p&gt;Issue eleven... &lt;/p&gt;
>
> If I don't change the angle brackets it gets worse:
>
>     &amp;lt;p&amp;gt;Issue eleven...
>
> And I can only assume that gets incrementally worse with every  
> "update" click.
>
> What am I doing wrong?
>
>
> -- Clive
>
> --
> ------------------------------
> Lasso Support: http://support.omnipilot.com/
> Search the list archives: http://www.listsearch.com/lassotalk.lasso
> Manage your list subscription: http://www.listsearch.com/ 
> lassotalk.lasso?manage
>


--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

Clive Bruton
In reply to this post by Clive Bruton

On 15 Sep 2005, at 05:57, Adam Randall wrote:

> It's a simple bug in the DatabaseBrowser.LassoApp. If you want a  
> fix, please join the beta program.

How?


-- Clive

--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage
Reply | Threaded
Open this post in threaded view
|

Re: Encoding, Unicode and Lasso database broswer app

jc-15
In reply to this post by Clive Bruton
> Anyone who wants to join the beta test can do so by sending an  
> email to <[hidden email]>.  You will be added to the beta  
> talk list and receive instructions for how to download the latest  
> beta.

By reading old messages... :-)

HDB
JC
15 sep 2005 kl. 11.23 skrev Clive Bruton:

>
> On 15 Sep 2005, at 05:57, Adam Randall wrote:
>
>
>> It's a simple bug in the DatabaseBrowser.LassoApp. If you want a  
>> fix, please join the beta program.
>>
>
> How?
>
>
> -- Clive
>
> --
> ------------------------------
> Lasso Support: http://support.omnipilot.com/
> Search the list archives: http://www.listsearch.com/lassotalk.lasso
> Manage your list subscription: http://www.listsearch.com/ 
> lassotalk.lasso?manage
>


--
------------------------------
Lasso Support: http://support.omnipilot.com/
Search the list archives: http://www.listsearch.com/lassotalk.lasso
Manage your list subscription:  
http://www.listsearch.com/lassotalk.lasso?manage