Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

bienvenue
Hi List

As mentionned in previous post (http://www.lassotalk.com/Lasso-8-6-CentOS-6-5-Lasso-plays-maths-with-comma-instead-of-dot.lasso?276505)
we are migrating from Mac OS X / Lasso 8.5 to VM / Cent OS 6.5 / Lasso 8.6.

For the purpose, we have dumped mySQL data from old server to be imported to mySQL in the new VM.
No problem… But …

Special Characters, as the euro symbol (€), apostophes and double quotes (the typographic ones) do not display as expected in the client browser.

- The symbol is correctly stored in the databases table, and readable in a database browser - as sequel pro -, as imported from the mySql dump.
On screen it appears as a "gremlins" symbol
- When typing in a form the euro symbol, and submitting, we retrieve it as an unreadable char in the table.

We did search on lasso talk post about similar problems shared on list.
No way to solve the matter.

We tried and checked followings among other points :
- verify encoding from mySQL - utf8 -  to lasso file - utf8 with BOM
- in lasso admin setting "Use MySQL 4.1/5.x Character Sets" to yes
- in queries, we tested to add or remove the - table inline argument, supposed to determine data encoding.
- we tried to add the "SET NAMES" command in mySQL quieries to control data encoding in results
- …

Maybe is there an issue about euro symbol or utf8 encoding and our setup ?
Where ? OS ? Connector ? Apache (web server in operation) ?

If you ever have hints about that, please share.
Regards
---
Jean-Pierre Kocherhans
E SYSTEMES
http://www.e-systemes.com

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

stevepiercy
I break down these Lasso 8 encoding issues step-by-step.  Each
step adds another layer to check encoding and character set issues.

1.  Put this code into a Lasso file saved with encoding of UTF-8
with BOM so that it is the only code in the file, then load the
file: ['€']
2.  It is possible that € cannot be displayed by the font used
in your styles or character set.  Add any styles and character
sets to the HTML and CSS from the previous step.
3.  Create a simple form with a single input and a submit button
that displays the submitted value.  Do not connect it to a database.
4.  Manually insert the value into a record in the database, and
use an inline to retrieve the record and display the value.
5.  Use an inline to write the value to a record in the database.

Once you go through the process, you will isolate where it
breaks down and be able to target possible options to solve the issue.

--steve


On 12/24/14 at 12:23 PM, [hidden email] (bienvenue) pronounced:

>Hi List
>
>As mentionned in previous post (http://www.lassotalk.com/Lasso-8-6-CentOS-6-5-Lasso-plays-maths-with-comma-instead-of-
>dot.lasso?276505)
>we are migrating from Mac OS X / Lasso 8.5 to VM / Cent OS 6.5 / Lasso 8.6.
>
>For the purpose, we have dumped mySQL data from old server to
>be imported to mySQL in the new VM.
>No problem… But …
>
>Special Characters, as the euro symbol (€), apostophes and
>double quotes (the typographic ones) do not display as expected
>in the client browser.
>
>- The symbol is correctly stored in the databases table, and
>readable in a database browser - as sequel pro -, as imported
>from the mySql dump.
>On screen it appears as a "gremlins" symbol
>- When typing in a form the euro symbol, and submitting, we
>retrieve it as an unreadable char in the table.
>
>We did search on lasso talk post about similar problems shared on list.
>No way to solve the matter.
>
>We tried and checked followings among other points :
>- verify encoding from mySQL - utf8 -  to lasso file - utf8 with BOM
>- in lasso admin setting "Use MySQL 4.1/5.x Character Sets" to yes
>- in queries, we tested to add or remove the - table inline
>argument, supposed to determine data encoding.
>- we tried to add the "SET NAMES" command in mySQL quieries to
>control data encoding in results
>- …
>
>Maybe is there an issue about euro symbol or utf8 encoding and our setup ?
>Where ? OS ? Connector ? Apache (web server in operation) ?
>
>If you ever have hints about that, please share.
>Regards
>---
>Jean-Pierre Kocherhans
>E SYSTEMES
>http://www.e-systemes.com
>
>#############################################################
>
>This message is sent to you because you are subscribed to
>the mailing list Lasso [hidden email]
>Official list archives available at http://www.lassotalk.com
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Steve Piercy              Website Builder              Soquel, CA
<[hidden email]>               <http://www.StevePiercy.com/>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

bienvenue
So, we did folow your step by step test process Steve.

1, 2, 3 went okay.
From point #4 it goes wrong, as in our apps and websites.

Retrieving and displaying the manually inserted records produce a gremlin on screen
Updating a record from a form with euro symbol produce a gremlin in the record…

We guess the problem is about the lasso mySQL connector.
Do not see anything we can do…

So, we wrote a patch for a kind of "active inline" we use on the top of our controllers and model in the apps.
It consist in HTML encoding / decoding while saving / retrieving records.
It works, while in a not satisfying way, for new records…
Not a real solution.

Regards
---
Jean-Pierre Kocherhans
E SYSTEMES
http://www.e-systemes.com


Le 24 déc. 2014 à 12:58, Steve Piercy - Website Builder <[hidden email]> a écrit :

> I break down these Lasso 8 encoding issues step-by-step.  Each step adds another layer to check encoding and character set issues.
>
> 1.  Put this code into a Lasso file saved with encoding of UTF-8 with BOM so that it is the only code in the file, then load the file: ['€']
> 2.  It is possible that € cannot be displayed by the font used in your styles or character set.  Add any styles and character sets to the HTML and CSS from the previous step.
> 3.  Create a simple form with a single input and a submit button that displays the submitted value.  Do not connect it to a database.
> 4.  Manually insert the value into a record in the database, and use an inline to retrieve the record and display the value.
> 5.  Use an inline to write the value to a record in the database.
>
> Once you go through the process, you will isolate where it breaks down and be able to target possible options to solve the issue.
>
> --steve
>
>
> On 12/24/14 at 12:23 PM, [hidden email] (bienvenue) pronounced:
>
>> Hi List
>>
>> As mentionned in previous post (http://www.lassotalk.com/Lasso-8-6-CentOS-6-5-Lasso-plays-maths-with-comma-instead-of-
>> dot.lasso?276505)
>> we are migrating from Mac OS X / Lasso 8.5 to VM / Cent OS 6.5 / Lasso 8.6.
>>
>> For the purpose, we have dumped mySQL data from old server to be imported to mySQL in the new VM.
>> No problem… But …
>>
>> Special Characters, as the euro symbol (€), apostophes and double quotes (the typographic ones) do not display as expected in the client browser.
>>
>> ( . . . )
>>
>> We tried and checked followings among other points :
>> - verify encoding from mySQL - utf8 -  to lasso file - utf8 with BOM
>> - in lasso admin setting "Use MySQL 4.1/5.x Character Sets" to yes
>> - in queries, we tested to add or remove the - table inline argument, supposed to determine data encoding.
>> - we tried to add the "SET NAMES" command in mySQL quieries to control data encoding in results
>> - …
>>
>> Maybe is there an issue about euro symbol or utf8 encoding and our setup ?
>> Where ? OS ? Connector ? Apache (web server in operation) ?
>>
>> If you ever have hints about that, please share.
>> Regards
>> --


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

stevepiercy
What form of connection do you use in your Lasso inlines?  -host
or regular?

If you use -host connections, you should specify the
-tableencoding='utf-8' parameter.
http://lasso.2283332.n4.nabble.com/Lasso-MySQL-and-Encoding-tp4636623p4636630.html

If you do not use -host connections, then the encoding is
determined by the encoding of the table specified in the -table parameter.

That's the most likely suspect.  There could be a couple of
other places, including the encoding you specify in Lasso
SiteAdmin (you already checked that across Data Sources >
Hosts/Databases/Tables) and the table encoding itself in MySQL
(I think you said you checked that, too).

--steve


On 12/24/14 at 2:33 PM, [hidden email] (bienvenue) pronounced:

>So, we did folow your step by step test process Steve.
>
>1, 2, 3 went okay.
>From point #4 it goes wrong, as in our apps and websites.
>
>Retrieving and displaying the manually inserted records produce a gremlin on screen
>Updating a record from a form with euro symbol produce a gremlin in the record…
>
>We guess the problem is about the lasso mySQL connector.
>Do not see anything we can do…
>
>So, we wrote a patch for a kind of "active inline" we use on
>the top of our controllers and model in the apps.
>It consist in HTML encoding / decoding while saving / retrieving records.
>It works, while in a not satisfying way, for new records…
>Not a real solution.
>
>Regards
>---
>Jean-Pierre Kocherhans
>E SYSTEMES
>http://www.e-systemes.com
>
>
>Le 24 déc. 2014 à 12:58, Steve Piercy - Website Builder <[hidden email]> a écrit :
>
>>I break down these Lasso 8 encoding issues step-by-step.  Each step adds another layer
>to check encoding and character set issues.
>>
>>1.  Put this code into a Lasso file saved with encoding of UTF-8 with BOM so that it is
>the only code in the file, then load the file: ['€']
>>2.  It is possible that € cannot be displayed by the font used in your styles or
>character set.  Add any styles and character sets to the HTML
>and CSS from the previous step.
>>3.  Create a simple form with a single input and a submit button that displays the
>submitted value.  Do not connect it to a database.
>>4.  Manually insert the value into a record in the database, and use an inline to
>retrieve the record and display the value.
>>5.  Use an inline to write the value to a record in the database.
>>
>>Once you go through the process, you will isolate where it breaks down and be able to
>target possible options to solve the issue.
>>
>>--steve
>>
>>
>>On 12/24/14 at 12:23 PM, [hidden email] (bienvenue) pronounced:
>>
>>> Hi List
>>>   As mentionned in previous post
>(http://www.lassotalk.com/Lasso-8-6-CentOS-6-5-Lasso-plays-maths-with-comma-instead-of-
>>> dot.lasso?276505)
>>> we are migrating from Mac OS X / Lasso 8.5 to VM / Cent OS 6.5 / Lasso 8.6.
>>>   For the purpose, we have dumped mySQL data from old server
>>>to be imported to mySQL in
>the new VM.
>>> No problem… But …
>>>   Special Characters, as the euro symbol (€), apostophes
>>>and double quotes (the
>typographic ones) do not display as expected in the client browser.
>>>   ( . . . )
>>>   We tried and checked followings among other points :
>>> - verify encoding from mySQL - utf8 -  to lasso file - utf8 with BOM
>>> - in lasso admin setting "Use MySQL 4.1/5.x Character Sets" to yes
>>> - in queries, we tested to add or remove the - table inline argument, supposed to
>determine data encoding.
>>> - we tried to add the "SET NAMES" command in mySQL quieries to control data encoding
>in results
>>> - …
>>>   Maybe is there an issue about euro symbol or utf8 encoding
>>>and our setup ?
>>> Where ? OS ? Connector ? Apache (web server in operation) ?
>>>   If you ever have hints about that, please share.
>>> Regards
>>> --
>
>
>#############################################################
>
>This message is sent to you because you are subscribed to
>the mailing list Lasso [hidden email]
>Official list archives available at http://www.lassotalk.com
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Steve Piercy              Website Builder              Soquel, CA
<[hidden email]>               <http://www.StevePiercy.com/>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

bienvenue
We just tried the -host tip with -tableencoding='utf-8'.
Same result : char encoding is wrong.

At this point it really seems to be a bug on CentOS 6.5 / Lasso 8.6 connector…

We have left in our patch for the moment, though we have to find something else in case our custumers have the idea to record some chineese or russian !-)

We are leaving for now (XMas…). We'll re-check everything later on.

Thanks Steve for your help.
If any of you find out something about that ?

Happy XMAS everyone !
Regards
JP
---
Jean-Pierre Kocherhans
E SYSTEMES
http://www.e-systemes.com
---

Le 24 déc. 2014 à 15:30, Steve Piercy - Website Builder <[hidden email]> a écrit :

> What form of connection do you use in your Lasso inlines?  -host or regular?
>
> If you use -host connections, you should specify the -tableencoding='utf-8' parameter.
> http://lasso.2283332.n4.nabble.com/Lasso-MySQL-and-Encoding-tp4636623p4636630.html
>
> If you do not use -host connections, then the encoding is determined by the encoding of the table specified in the -table parameter.
>
> That's the most likely suspect.  There could be a couple of other places, including the encoding you specify in Lasso SiteAdmin (you already checked that across Data Sources > Hosts/Databases/Tables) and the table encoding itself in MySQL (I think you said you checked that, too).
>
> --steve
>
>
> On 12/24/14 at 2:33 PM, [hidden email] (bienvenue) pronounced:
>
>> So, we did folow your step by step test process Steve.
>>
>> 1, 2, 3 went okay.
>> From point #4 it goes wrong, as in our apps and websites.
>>
>> Retrieving and displaying the manually inserted records produce a gremlin on screen
>> Updating a record from a form with euro symbol produce a gremlin in the record…
>>
>> We guess the problem is about the lasso mySQL connector.
>> Do not see anything we can do…
>>
>> So, we wrote a patch for a kind of "active inline" we use on the top of our controllers and model in the apps.
>> It consist in HTML encoding / decoding while saving / retrieving records.
>> It works, while in a not satisfying way, for new records…
>> Not a real solution.
>>
>> Regards
>> ---
>> Jean-Pierre Kocherhans
>> E SYSTEMES
>> http://www.e-systemes.com
>>
>>
>> Le 24 déc. 2014 à 12:58, Steve Piercy - Website Builder <[hidden email]> a écrit :
>>
>>> I break down these Lasso 8 encoding issues step-by-step.  Each step adds another layer
>> to check encoding and character set issues.
>>>
>>> 1.  Put this code into a Lasso file saved with encoding of UTF-8 with BOM so that it is
>> the only code in the file, then load the file: ['€']
>>> 2.  It is possible that € cannot be displayed by the font used in your styles or
>> character set.  Add any styles and character sets to the HTML and CSS from the previous step.
>>> 3.  Create a simple form with a single input and a submit button that displays the
>> submitted value.  Do not connect it to a database.
>>> 4.  Manually insert the value into a record in the database, and use an inline to
>> retrieve the record and display the value.
>>> 5.  Use an inline to write the value to a record in the database.
>>>
>>> Once you go through the process, you will isolate where it breaks down and be able to
>> target possible options to solve the issue.
>>>
>>> --steve
>>>
>>>
>>> On 12/24/14 at 12:23 PM, [hidden email] (bienvenue) pronounced:
>>>
>>>> Hi List
>>>>  As mentionned in previous post
>> (http://www.lassotalk.com/Lasso-8-6-CentOS-6-5-Lasso-plays-maths-with-comma-instead-of-
>>>> dot.lasso?276505)
>>>> we are migrating from Mac OS X / Lasso 8.5 to VM / Cent OS 6.5 / Lasso 8.6.
>>>>  For the purpose, we have dumped mySQL data from old server to be imported to mySQL in
>> the new VM.
>>>> No problem… But …
>>>>  Special Characters, as the euro symbol (€), apostophes and double quotes (the
>> typographic ones) do not display as expected in the client browser.
>>>>  ( . . . )
>>>>  We tried and checked followings among other points :
>>>> - verify encoding from mySQL - utf8 -  to lasso file - utf8 with BOM
>>>> - in lasso admin setting "Use MySQL 4.1/5.x Character Sets" to yes
>>>> - in queries, we tested to add or remove the - table inline argument, supposed to
>> determine data encoding.
>>>> - we tried to add the "SET NAMES" command in mySQL quieries to control data encoding
>> in results
>>>> - …
>>>>  Maybe is there an issue about euro symbol or utf8 encoding and our setup ?
>>>> Where ? OS ? Connector ? Apache (web server in operation) ?
>>>>  If you ever have hints about that, please share.
>>>> Regards
>>>> --
>>
>>
>> #############################################################
>>
>> This message is sent to you because you are subscribed to
>> the mailing list Lasso [hidden email]
>> Official list archives available at http://www.lassotalk.com
>> To unsubscribe, E-mail to: <[hidden email]>
>> Send administrative queries to  <[hidden email]>
>
> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> Steve Piercy              Website Builder              Soquel, CA
> <[hidden email]>               <http://www.StevePiercy.com/>
>
>
> #############################################################
>
> This message is sent to you because you are subscribed to
> the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

stevepiercy
It's highly unlikely that there's a bug in Lasso's MySQL
connector.  We're using the same connector, and it works for me.

Try the following.

Create a new table using the following SQL statement:

     CREATE TABLE `<database_name>`.`encoding` (
         `id` int UNSIGNED NOT NULL AUTO_INCREMENT,
         `value` varchar(255),
         PRIMARY KEY (`id`),
         UNIQUE `id` USING BTREE (`id`) comment '',
         INDEX `value` USING BTREE (`value`) comment ''
     ) ENGINE=MyISAM DEFAULT CHARSET=utf8

This table uses a character set of UTF-8.  If CHARSET is
omitted, it will use the database's default.

Then use the following code to insert a record with a €, then
retrieve it.  Save the file with encoding of UTF-8 with BOM.  
This code works for me.

If my example works for you, then the problem is in your code.

If my example fails for you, then you have database encoding not
set to use UTF-8 either in MySQL (see above CHARSET comment) or
in Lasso SiteAdmin, under:

Setup > Site > Lasso Settings > Default Page Encoding > Encoding
Setup > Datasources > Hosts > Host Detail
Setup > Datasources > Tables > Table Detail

[
     // configure these values
     local(
         'h' = 'localhost',
         'd' = 'mydatabase',
         't' = 'mytable',
         'u' = 'myusername',
         'p' = 'mypassword',
     );
     inline(-host=array(
         -datasource='mysqlds',
         -name=#h,
         -username=#u,
         -password=#p,
         -tableencoding='utf-8'),
     -database=#d,
     // toggle between -sql and traditional Lasso inline statements
     -SQL='insert into `'+#d+'`.`'+#t+'` (`value`) values ("€");'
     //-table=#t,'value'='€',-add
     );
         local('kfv') = keyfield_value;
     /inline;

     inline(-host=array(
         -datasource='mysqlds',
         -name=#h,
         -username=#u,
         -password=#p,
         -tableencoding='utf-8'),
     -database=#d,
     -SQL='select * from `'+#d+'`.`'+#t+'` where id = '+#kfv,
     -maxrecords=1
     );
     action_statement;
     '<br>';
     error_currenterror;
     '<br><br>Found Count is: '+found_count+'<br>';
         records;
     field('value');
     '<br><br>';
         /records;
     /inline;
]

--steve


On 12/24/14 at 5:08 PM, [hidden email] (bienvenue) pronounced:

>We just tried the -host tip with -tableencoding='utf-8'.
>Same result : char encoding is wrong.
>
>At this point it really seems to be a bug on CentOS 6.5 / Lasso 8.6 connector…
>
>We have left in our patch for the moment, though we have to
>find something else in case our custumers have the idea to
>record some chineese or russian !-)
>
>We are leaving for now (XMas…). We'll re-check everything later on.
>
>Thanks Steve for your help.
>If any of you find out something about that ?
>
>Happy XMAS everyone !
>Regards
>JP
>---
>Jean-Pierre Kocherhans
>E SYSTEMES
>http://www.e-systemes.com
>---
>
>Le 24 déc. 2014 à 15:30, Steve Piercy - Website Builder <[hidden email]> a écrit :
>
>>What form of connection do you use in your Lasso inlines?  -host or regular?
>>
>>If you use -host connections, you should specify the -tableencoding='utf-8' parameter.
>>http://lasso.2283332.n4.nabble.com/Lasso-MySQL-and-Encoding-tp4636623p4636630.html
>>
>>If you do not use -host connections, then the encoding is determined by the encoding of
>the table specified in the -table parameter.
>>
>>That's the most likely suspect.  There could be a couple of other places, including the
>encoding you specify in Lasso SiteAdmin (you already checked
>that across Data Sources > Hosts/Databases/Tables) and the
>table encoding itself in MySQL (I think you said you checked
>that, too).
>>
>>--steve
>>
>>
>>On 12/24/14 at 2:33 PM, [hidden email] (bienvenue) pronounced:
>>
>>> So, we did folow your step by step test process Steve.
>>>   1, 2, 3 went okay.
>>> From point #4 it goes wrong, as in our apps and websites.
>>>   Retrieving and displaying the manually inserted records
>>>produce a gremlin on screen
>>> Updating a record from a form with euro symbol produce a gremlin in the record…
>>>   We guess the problem is about the lasso mySQL connector.
>>> Do not see anything we can do…
>>>   So, we wrote a patch for a kind of "active inline" we use
>>>on the top of our
>controllers and model in the apps.
>>> It consist in HTML encoding / decoding while saving / retrieving records.
>>> It works, while in a not satisfying way, for new records…
>>> Not a real solution.
>>>   Regards
>>> ---
>>> Jean-Pierre Kocherhans
>>> E SYSTEMES
>>> http://www.e-systemes.com
>>>    Le 24 déc. 2014 à 12:58, Steve Piercy - Website Builder
>>><[hidden email]> a écrit
>:
>>>
>>>> I break down these Lasso 8 encoding issues step-by-step.  Each step adds another
>layer
>>> to check encoding and character set issues.
>>>>   1.  Put this code into a Lasso file saved with encoding
>>>>of UTF-8 with BOM so that it
>is
>>> the only code in the file, then load the file: ['€']
>>>> 2.  It is possible that € cannot be displayed by the font used in your styles or
>>> character set.  Add any styles and character sets to the HTML and CSS from the
>previous step.
>>>> 3.  Create a simple form with a single input and a submit button that displays the
>>> submitted value.  Do not connect it to a database.
>>>> 4.  Manually insert the value into a record in the database, and use an inline to
>>> retrieve the record and display the value.
>>>> 5.  Use an inline to write the value to a record in the database.
>>>>   Once you go through the process, you will isolate where
>>>>it breaks down and be able to
>>> target possible options to solve the issue.
>>>>   --steve
>>>>    On 12/24/14 at 12:23 PM, [hidden email]
>>>>(bienvenue) pronounced:
>>>>
>>>>> Hi List
>>>>>  As mentionned in previous post
>>>
>(http://www.lassotalk.com/Lasso-8-6-CentOS-6-5-Lasso-plays-maths-with-comma-instead-of-
>>>>> dot.lasso?276505)
>>>>> we are migrating from Mac OS X / Lasso 8.5 to VM / Cent OS 6.5 / Lasso 8.6.
>>>>>  For the purpose, we have dumped mySQL data from old server to be imported to mySQL
>in
>>> the new VM.
>>>>> No problem… But …
>>>>>  Special Characters, as the euro symbol (€), apostophes and double quotes (the
>>> typographic ones) do not display as expected in the client browser.
>>>>>  ( . . . )
>>>>>  We tried and checked followings among other points :
>>>>> - verify encoding from mySQL - utf8 -  to lasso file - utf8 with BOM
>>>>> - in lasso admin setting "Use MySQL 4.1/5.x Character Sets" to yes
>>>>> - in queries, we tested to add or remove the - table inline argument, supposed to
>>> determine data encoding.
>>>>> - we tried to add the "SET NAMES" command in mySQL quieries to control data encoding
>>> in results
>>>>> - …
>>>>>  Maybe is there an issue about euro symbol or utf8 encoding and our setup ?
>>>>> Where ? OS ? Connector ? Apache (web server in operation) ?
>>>>>  If you ever have hints about that, please share.
>>>>> Regards
>>>>> --
>>>    #############################################################
>>>   This message is sent to you because you are subscribed to
>>> the mailing list Lasso [hidden email]
>>> Official list archives available at http://www.lassotalk.com
>>> To unsubscribe, E-mail to: <[hidden email]>
>>> Send administrative queries to  <[hidden email]>
>>
>>-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>Steve Piercy              Website Builder              Soquel, CA
>><[hidden email]>               <http://www.StevePiercy.com/>
>>
>>
>>#############################################################
>>
>>This message is sent to you because you are subscribed to
>>the mailing list Lasso [hidden email]
>>Official list archives available at http://www.lassotalk.com
>>To unsubscribe, E-mail to: <[hidden email]>
>>Send administrative queries to  <[hidden email]>
>
>
>#############################################################
>
>This message is sent to you because you are subscribed to
>the mailing list Lasso [hidden email]
>Official list archives available at http://www.lassotalk.com
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Steve Piercy              Website Builder              Soquel, CA
<[hidden email]>               <http://www.StevePiercy.com/>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

Jussi Hirvi-2
In reply to this post by bienvenue
On 24.12.2014 18.08, bienvenue wrote:
> We just tried the -host tip with -tableencoding='utf-8'.
> Same result : char encoding is wrong.

As the error seems to be somewhere in MySQL or Lasso, here is one thing
to check: the db and table encoding in MySQL.

Here is how you can do it on command-line. In case you are a MySQL
wizard, I hope this will be useful to others. :-)

// in mysql, select your db
use mydb;
// show charset for database
SHOW VARIABLES LIKE 'character\_set\_%';
// show charset for a table
SELECT CCSA.character_set_name FROM information_schema.`TABLES` T,
information_schema.`COLLATION_CHARACTER_SET_APPLICABILITY` CCSA WHERE
CCSA.collation_name = T.table_collation AND T.table_name = "mytable";

// change charset for the whole database
alter database mydb default character set = utf8;
// change charset for a table
ALTER TABLE mytable CONVERT TO CHARACTER SET charset_name;


- Jussi

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Lasso 8.6, CentOS 6.6, MySQL and euro symbol problem

bienvenue
In reply to this post by stevepiercy
Hi Steve

Many thanks for your great help.

We did test your code, which works…

Then we re-re-…-re-checked lasso site admin setup, concerning encoding.
In setUp > Data Sources > Databases > Tables
We noticed that under "Table Detail" Encoding was set to Latin-1 (WTF)

So, back to setUp > Data Sources > Databases web, did a "Table Batch Change" for each and every one, changing encoding to UTF-8.
That's ok now...

That fuckin' Lasso Admin is definitly tricky.
All that precious time lost, clicking through tabs and again…
Why those Databases and tables set in mySQL as UTF-8 are not symply served by Lasso as they are ?
Or why default encoding is not UTF-8 ?

Compared to PHP setup with php.ini, what a loss of time and head mess…
Very hard to explain to my team, as to myself ! Even if I am an early Lasso user ...

Well. Thanks again for your help Steve and Jussi
Regards
---
Jean-Pierre Kocherhans
E SYSTEMES
http://www.e-systemes.com

Le 24 déc. 2014 à 22:55, Steve Piercy - Website Builder <[hidden email]> a écrit :

> It's highly unlikely that there's a bug in Lasso's MySQL connector.  We're using the same connector, and it works for me.
>
> Try the following.
>
> Create a new table using the following SQL statement:
>
>    CREATE TABLE `<database_name>`.`encoding` (
>        `id` int UNSIGNED NOT NULL AUTO_INCREMENT,
>        `value` varchar(255),
>        PRIMARY KEY (`id`),
>        UNIQUE `id` USING BTREE (`id`) comment '',
>        INDEX `value` USING BTREE (`value`) comment ''
>    ) ENGINE=MyISAM DEFAULT CHARSET=utf8
>
> This table uses a character set of UTF-8.  If CHARSET is omitted, it will use the database's default.
>
> Then use the following code to insert a record with a €, then retrieve it.  Save the file with encoding of UTF-8 with BOM.  This code works for me.
>
> If my example works for you, then the problem is in your code.
>
> If my example fails for you, then you have database encoding not set to use UTF-8 either in MySQL (see above CHARSET comment) or in Lasso SiteAdmin, under:
>
> Setup > Site > Lasso Settings > Default Page Encoding > Encoding
> Setup > Datasources > Hosts > Host Detail
> Setup > Datasources > Tables > Table Detail
>
> [
>    // configure these values
>    local(
>        'h' = 'localhost',
>        'd' = 'mydatabase',
>        't' = 'mytable',
>        'u' = 'myusername',
>        'p' = 'mypassword',
>    );
>    inline(-host=array(
>        -datasource='mysqlds',
>        -name=#h,
>        -username=#u,
>        -password=#p,
>        -tableencoding='utf-8'),
>    -database=#d,
>    // toggle between -sql and traditional Lasso inline statements
>    -SQL='insert into `'+#d+'`.`'+#t+'` (`value`) values ("€");'
>    //-table=#t,'value'='€',-add
>    );
>        local('kfv') = keyfield_value;
>    /inline;
>
>    inline(-host=array(
>        -datasource='mysqlds',
>        -name=#h,
>        -username=#u,
>        -password=#p,
>        -tableencoding='utf-8'),
>    -database=#d,
>    -SQL='select * from `'+#d+'`.`'+#t+'` where id = '+#kfv,
>    -maxrecords=1
>    );
>    action_statement;
>    '<br>';
>    error_currenterror;
>    '<br><br>Found Count is: '+found_count+'<br>';
>        records;
>    field('value');
>    '<br><br>';
>        /records;
>    /inline;
> ]
>
> --steve
>
>
> On 12/24/14 at 5:08 PM, [hidden email] (bienvenue) pronounced:
>
>> We just tried the -host tip with -tableencoding='utf-8'.
>> Same result : char encoding is wrong.
>>
>> At this point it really seems to be a bug on CentOS 6.5 / Lasso 8.6 connector…
>>
>> We have left in our patch for the moment, though we have to find something else in case our custumers have the idea to record some chineese or russian !-)
>>
>> We are leaving for now (XMas…). We'll re-check everything later on.
>>
>> Thanks Steve for your help.
>> If any of you find out something about that ?
>>
>> Happy XMAS everyone !
>> Regards
>> JP
>> ---
>> Jean-Pierre Kocherhans
>> E SYSTEMES
>> http://www.e-systemes.com
>> ---
>>
>> Le 24 déc. 2014 à 15:30, Steve Piercy - Website Builder <[hidden email]> a écrit :
>>
>>> What form of connection do you use in your Lasso inlines?  -host or regular?
>>>
>>> If you use -host connections, you should specify the -tableencoding='utf-8' parameter.
>>> http://lasso.2283332.n4.nabble.com/Lasso-MySQL-and-Encoding-tp4636623p4636630.html
>>>
>>> If you do not use -host connections, then the encoding is determined by the encoding of
>> the table specified in the -table parameter.
>>>
>>> That's the most likely suspect.  There could be a couple of other places, including the
>> encoding you specify in Lasso SiteAdmin (you already checked that across Data Sources > Hosts/Databases/Tables) and the table encoding itself in MySQL (I think you said you checked that, too).
>>>
>>> --steve
>>>
>>>
>>> On 12/24/14 at 2:33 PM, [hidden email] (bienvenue) pronounced:
>>>
>>>> So, we did folow your step by step test process Steve.
>>>>  1, 2, 3 went okay.
>>>> From point #4 it goes wrong, as in our apps and websites.
>>>>  Retrieving and displaying the manually inserted records produce a gremlin on screen
>>>> Updating a record from a form with euro symbol produce a gremlin in the record…
>>>>  We guess the problem is about the lasso mySQL connector.
>>>> Do not see anything we can do…
>>>>  So, we wrote a patch for a kind of "active inline" we use on the top of our
>> controllers and model in the apps.
>>>> It consist in HTML encoding / decoding while saving / retrieving records.
>>>> It works, while in a not satisfying way, for new records…
>>>> Not a real solution.
>>>>  Regards
>>>> ---
>>>> Jean-Pierre Kocherhans
>>>> E SYSTEMES
>>>> http://www.e-systemes.com
>>>>   Le 24 déc. 2014 à 12:58, Steve Piercy - Website Builder <[hidden email]> a écrit
>> :
>>>>
>>>>> I break down these Lasso 8 encoding issues step-by-step.  Each step adds another
>> layer
>>>> to check encoding and character set issues.
>>>>>  1.  Put this code into a Lasso file saved with encoding of UTF-8 with BOM so that it
>> is
>>>> the only code in the file, then load the file: ['€']
>>>>> 2.  It is possible that € cannot be displayed by the font used in your styles or
>>>> character set.  Add any styles and character sets to the HTML and CSS from the
>> previous step.
>>>>> 3.  Create a simple form with a single input and a submit button that displays the
>>>> submitted value.  Do not connect it to a database.
>>>>> 4.  Manually insert the value into a record in the database, and use an inline to
>>>> retrieve the record and display the value.
>>>>> 5.  Use an inline to write the value to a record in the database.
>>>>>  Once you go through the process, you will isolate where it breaks down and be able to
>>>> target possible options to solve the issue.
>>>>>  --steve
>>>>>   On 12/24/14 at 12:23 PM, [hidden email] (bienvenue) pronounced:
>>>>>
>>>>>> Hi List
>>>>>> As mentionned in previous post
>>>>
>> (http://www.lassotalk.com/Lasso-8-6-CentOS-6-5-Lasso-plays-maths-with-comma-instead-of-
>>>>>> dot.lasso?276505)
>>>>>> we are migrating from Mac OS X / Lasso 8.5 to VM / Cent OS 6.5 / Lasso 8.6.
>>>>>> For the purpose, we have dumped mySQL data from old server to be imported to mySQL
>> in
>>>> the new VM.
>>>>>> No problem… But …
>>>>>> Special Characters, as the euro symbol (€), apostophes and double quotes (the
>>>> typographic ones) do not display as expected in the client browser.
>>>>>> ( . . . )
>>>>>> We tried and checked followings among other points :
>>>>>> - verify encoding from mySQL - utf8 -  to lasso file - utf8 with BOM
>>>>>> - in lasso admin setting "Use MySQL 4.1/5.x Character Sets" to yes
>>>>>> - in queries, we tested to add or remove the - table inline argument, supposed to
>>>> determine data encoding.
>>>>>> - we tried to add the "SET NAMES" command in mySQL quieries to control data encoding
>>>> in results
>>>>>> - …
>>>>>> Maybe is there an issue about euro symbol or utf8 encoding and our setup ?
>>>>>> Where ? OS ? Connector ? Apache (web server in operation) ?
>>>>>> If you ever have hints about that, please share.
>>>>>> Regards
>>>>>> --
>>>>   #############################################################
>>>>  This message is sent to you because you are subscribed to
>>>> the mailing list Lasso [hidden email]
>>>> Official list archives available at http://www.lassotalk.com
>>>> To unsubscribe, E-mail to: <[hidden email]>
>>>> Send administrative queries to  <[hidden email]>
>>>
>>> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
>>> Steve Piercy              Website Builder              Soquel, CA
>>> <[hidden email]>               <http://www.StevePiercy.com/>
>>>
>>>
>>> #############################################################
>>>
>>> This message is sent to you because you are subscribed to
>>> the mailing list Lasso [hidden email]
>>> Official list archives available at http://www.lassotalk.com
>>> To unsubscribe, E-mail to: <[hidden email]>
>>> Send administrative queries to  <[hidden email]>
>>
>>
>> #############################################################
>>
>> This message is sent to you because you are subscribed to
>> the mailing list Lasso [hidden email]
>> Official list archives available at http://www.lassotalk.com
>> To unsubscribe, E-mail to: <[hidden email]>
>> Send administrative queries to  <[hidden email]>
>
> -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
> Steve Piercy              Website Builder              Soquel, CA
> <[hidden email]>               <http://www.StevePiercy.com/>
>
>
> #############################################################
>
> This message is sent to you because you are subscribed to
> the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>