Emoticon UTF-8 issues when adding rows to Mysql table

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
Lasso 9.3
Mysql 5.6.17 (In my OSX based tests. On the original server probably something else.)

A client attempted to add some text that was copied from an email conversation into a form that in turn was sent to a Mysql table using DS. Failed. Turns out it’s because there is an emoticon in the pasted text.

I could replicate this with a test case.

mydsobject -> keycolumn(::id) -> addrow((:
        ’mytextfield' = ' 😊'
))

To make sure it was not a strict DS issue I tried the same with a regular inline

inline(-Add,
        -Database = ’mydb',
        -Table = ’mytable',
        'mytextfield'= ' 😊'
) => {^
error_msg
^}


The error message I get is
Incorrect string value: '\xF0\x9F\x98\x8A' for column 'mytextfield' at row 1.

In real life the value is of course not hardcoded. It is fetched from web_request -> params
web_request -> param(’myvalue’) -> asstring.

When writing this far I figured I’d search for the error and see what came up. This is what I found:
https://mathiasbynens.be/notes/mysql-utf8mb4
Turns out that, despite popular belief, Mysql UTF-8 does not support UTF-8 in it’s full. It will deal with 1-3 byte chars but not 4 byte chars. And of course the emoticons are 4 byte.

I followed the instructions in the article above but only with partial success. No matter what, I could not get emoticons accepted when inserting them thru Lasso. Nor did I ever manage to create a record using Navicat. I was however able to get an emoticon added using Aureliums Querious and thru the CLI mysql command. Indicating that the Mysql settings indeed are correct now. And yes, if I use Querious or CLI mysql on a setup that has not been changed to utf8mb4 I get the same error as before.

Here are by the way the current (new) Mysql settings:
+--------------------------+--------------------+
| Variable_name            | Value              |
+--------------------------+--------------------+
| character_set_client     | utf8mb4            |
| character_set_connection | utf8mb4            |
| character_set_database   | utf8mb4            |
| character_set_filesystem | binary             |
| character_set_results    | utf8mb4            |
| character_set_server     | utf8mb4            |
| character_set_system     | utf8               |
| collation_connection     | utf8mb4_swedish_ci |
| collation_database       | utf8mb4_swedish_ci |
| collation_server         | utf8mb4_swedish_ci |
+--------------------------+——————————+

If I am correct then there’s a flaw in the Lasso Mysql connector preventing us from inserting the ever increasingly popular emoticons, and other 4 byte UTF chars, into our Mysql solutions. Would be nice to have this either confirmed and fixed or shown where I’m doing it wrong so I can fix my code.

I can only imagine that this will soon affect others than me since almost every communication done today thru Twitter, iMessage or youNameIt contains one or several emoticons. I did a quick scan of messages I received yesterday and counted 7 different containing one or more happy face, clap hand, heart etc…

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
15 feb 2015 kl. 13:05 skrev Jolle Carlestam <[hidden email]>:

> If I am correct then there’s a flaw in the Lasso Mysql connector preventing us from inserting the ever increasingly popular emoticons, and other 4 byte UTF chars, into our Mysql solutions. Would be nice to have this either confirmed and fixed or shown where I’m doing it wrong so I can fix my code.

After I sent this I also added is as a bug report to Rhino trac. And after hitting submit there I realized that, Hey, this will fail since I included the emoticon in the email and submission. Lassosoft will not be able to save it.

But, no error was reported to me. Not knowing how Lassosoft deals with emails to the list or trac reports the following is pure speculation. But IF they save it in a DB somewhere, and IF they use Lasso 9.X for it. Then it might be that there is a way to do it for me and us as well.


….Ah, no. Now I've checked in Lassotalk. My message is truncated in there right where the emoticon was supposed to be. That is at least another error than I had. In my case, no record was added at all. In Lassotalk the record was created but cut off at the place of the emoticon.

http://www.lassotalk.com/Emoticon-UTF-8-issues-when-adding-rows-to-Mysql-table.lasso?276935

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Steffan A. Cline
Your email came to me with the emoji in it.

I can see where if you had issues, the LassoTalk archive would have been
truncated too.

Have you tried manually inserting that data into MySQL from the CLI to
make sure it's Lasso at fault?


Thanks,
Steffan

---------------------------------------------------------------
T E L  6 0 2 . 7 9 3 . 0 0 1 4 | F A X  6 0 2 . 9 7 1 . 1 6 9 4
Steffan A. Cline   [hidden email]
http://www.ExecuChoice.net                 Phoenix, Arizona USA
                 
---------------------------------------------------------------






On 2/15/15, 5:27 AM, "Jolle Carlestam" <[hidden email]> wrote:

>15 feb 2015 kl. 13:05 skrev Jolle Carlestam <[hidden email]>:
>
>> If I am correct then there¹s a flaw in the Lasso Mysql connector
>>preventing us from inserting the ever increasingly popular emoticons,
>>and other 4 byte UTF chars, into our Mysql solutions. Would be nice to
>>have this either confirmed and fixed or shown where I¹m doing it wrong
>>so I can fix my code.
>
>After I sent this I also added is as a bug report to Rhino trac. And
>after hitting submit there I realized that, Hey, this will fail since I
>included the emoticon in the email and submission. Lassosoft will not be
>able to save it.
>
>But, no error was reported to me. Not knowing how Lassosoft deals with
>emails to the list or trac reports the following is pure speculation. But
>IF they save it in a DB somewhere, and IF they use Lasso 9.X for it. Then
>it might be that there is a way to do it for me and us as well.
>
>
>Š.Ah, no. Now I've checked in Lassotalk. My message is truncated in there
>right where the emoticon was supposed to be. That is at least another
>error than I had. In my case, no record was added at all. In Lassotalk
>the record was created but cut off at the place of the emoticon.
>
>http://www.lassotalk.com/Emoticon-UTF-8-issues-when-adding-rows-to-Mysql-t
>able.lasso?276935
>
>HDB
>Jolle
>
>#############################################################
>
>This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
>Official list archives available at http://www.lassotalk.com
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>
>



#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Steffan A. Cline
Sorry… Missed the bottom of your email where you said you tested from the
CLI and Querious.

I could have sworn that I read some time ago that Kyle had updated
everything to handle up to UTF-16.

I'll have to look back for that.


Thanks,
Steffan

---------------------------------------------------------------
T E L  6 0 2 . 7 9 3 . 0 0 1 4 | F A X  6 0 2 . 9 7 1 . 1 6 9 4
Steffan A. Cline   [hidden email]
http://www.ExecuChoice.net                 Phoenix, Arizona USA
                 
---------------------------------------------------------------






On 2/15/15, 7:53 AM, "Steffan A. Cline" <[hidden email]> wrote:

>Your email came to me with the emoji in it.
>
>I can see where if you had issues, the LassoTalk archive would have been
>truncated too.
>
>Have you tried manually inserting that data into MySQL from the CLI to
>make sure it's Lasso at fault?
>
>
>Thanks,
>Steffan
>
>---------------------------------------------------------------
>T E L  6 0 2 . 7 9 3 . 0 0 1 4 | F A X  6 0 2 . 9 7 1 . 1 6 9 4
>Steffan A. Cline   [hidden email]
>http://www.ExecuChoice.net                 Phoenix, Arizona USA
>                  
>---------------------------------------------------------------
>
>
>
>
>
>
>On 2/15/15, 5:27 AM, "Jolle Carlestam" <[hidden email]> wrote:
>
>>15 feb 2015 kl. 13:05 skrev Jolle Carlestam <[hidden email]>:
>>
>>> If I am correct then there¹s a flaw in the Lasso Mysql connector
>>>preventing us from inserting the ever increasingly popular emoticons,
>>>and other 4 byte UTF chars, into our Mysql solutions. Would be nice to
>>>have this either confirmed and fixed or shown where I¹m doing it wrong
>>>so I can fix my code.
>>
>>After I sent this I also added is as a bug report to Rhino trac. And
>>after hitting submit there I realized that, Hey, this will fail since I
>>included the emoticon in the email and submission. Lassosoft will not be
>>able to save it.
>>
>>But, no error was reported to me. Not knowing how Lassosoft deals with
>>emails to the list or trac reports the following is pure speculation. But
>>IF they save it in a DB somewhere, and IF they use Lasso 9.X for it. Then
>>it might be that there is a way to do it for me and us as well.
>>
>>
>>�.Ah, no. Now I've checked in Lassotalk. My message is truncated in there
>>right where the emoticon was supposed to be. That is at least another
>>error than I had. In my case, no record was added at all. In Lassotalk
>>the record was created but cut off at the place of the emoticon.
>>
>>http://www.lassotalk.com/Emoticon-UTF-8-issues-when-adding-rows-to-Mysql-
>>t
>>able.lasso?276935
>>
>>HDB
>>Jolle
>>
>>#############################################################
>>
>>This message is sent to you because you are subscribed to
>>  the mailing list Lasso [hidden email]
>>Official list archives available at http://www.lassotalk.com
>>To unsubscribe, E-mail to: <[hidden email]>
>>Send administrative queries to  <[hidden email]>
>>
>
>
>
>#############################################################
>
>This message is sent to you because you are subscribed to
>  the mailing list Lasso [hidden email]
>Official list archives available at http://www.lassotalk.com
>To unsubscribe, E-mail to: <[hidden email]>
>Send administrative queries to  <[hidden email]>
>



#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
In reply to this post by Steffan A. Cline
My guess is that emails are sent separate from storing it in Lassotalk. That's why it came thru intact. The record in Lassotalk on the other hand is truncated.

HDB
Jolle

Sent from a mobile device. Any anomalies is due to Autocorrect.

> 15 feb 2015 kl. 15:53 skrev Steffan A. Cline <[hidden email]>:
>
> Your email came to me with the emoji in it.
>
> I can see where if you had issues, the LassoTalk archive would have been
> truncated too.

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
Some more investigation on this.
Turning the string to bytes before sending it to Mysql will not help as long as the receiving field is a text field. Same error.
However, it is possible to store the data provided it’s stored in a blob field.

This works:

myDS -> table(::mytable) -> keycolumn(::id) -> addrow((:
        'myblob' = ('😊') -> asbytes
))

It can then be retrieved like this:

with row in myDS -> sql("SELECT * FROM mytable") -> rows select #row(::myblob) + '<br />’

One caveat of course being that the data is no longer searchable. Would be ok for some situations but not other.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
In reply to this post by Jolle Carlestam-2
15 feb 2015 kl. 13:05 skrev Jolle Carlestam <[hidden email]>:

> there’s a flaw in the Lasso Mysql connector preventing us from inserting the ever increasingly popular emoticons, and other 4 byte UTF chars, into our Mysql solutions.

Pending that Lassosoft fixes this, any clues on how I can trap for 4 byte chars in user submitted data?
So that it would be possible to either remove/replace them or send back a good looking error message asking the user to clean up the data first.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

stevepiercy
On 2/16/15 at 10:26 PM, [hidden email] (Jolle Carlestam) pronounced:

>15 feb 2015 kl. 13:05 skrev Jolle Carlestam <[hidden email]>:
>
>>there’s a flaw in the Lasso Mysql connector preventing us from inserting the ever
>increasingly popular emoticons, and other 4 byte UTF chars, into our Mysql solutions.
>
>Pending that Lassosoft fixes this, any clues on how I can trap
>for 4 byte chars in user submitted data?
>So that it would be possible to either remove/replace them or
>send back a good looking error message asking the user to clean
>up the data first.

Super kludge method: store the value in a local, write the value
to the db, select the value, compare to local.  If different, reject.

There's probably a regex that will do what you want to avoid the
kludge.  To tehGooglez!

--steve

-- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Steve Piercy              Website Builder              Soquel, CA
<[hidden email]>               <http://www.StevePiercy.com/>


#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Bil Corry-3
In reply to this post by Jolle Carlestam-2
You might try sending:

    SET NAMES 'utf8mb4'

at the beginning of the connection with MySQL, then try your queries.  That
should set the connection to send/receive 4-byte UTF-8 characters.


- Bil



On Mon, Feb 16, 2015 at 8:38 AM, Jolle Carlestam <[hidden email]>
wrote:

> Some more investigation on this.
> Turning the string to bytes before sending it to Mysql will not help as
> long as the receiving field is a text field. Same error.
> However, it is possible to store the data provided it’s stored in a blob
> field.
>
> This works:
>
> myDS -> table(::mytable) -> keycolumn(::id) -> addrow((:
>         'myblob'        = ('😊') -> asbytes
> ))
>
> It can then be retrieved like this:
>
> with row in myDS -> sql("SELECT * FROM mytable") -> rows select
> #row(::myblob) + '<br />’
>
> One caveat of course being that the data is no longer searchable. Would be
> ok for some situations but not other.
>
> HDB
> Jolle
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>   the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Bil Corry-3
In reply to this post by Jolle Carlestam-2
You might be able to use regex, this is untested, but it may work:

     [\\x{10000}-\\x{10ffff}]


- Bil

On Mon, Feb 16, 2015 at 10:26 PM, Jolle Carlestam <[hidden email]>
wrote:

> 15 feb 2015 kl. 13:05 skrev Jolle Carlestam <[hidden email]>:
>
> > there’s a flaw in the Lasso Mysql connector preventing us from inserting
> the ever increasingly popular emoticons, and other 4 byte UTF chars, into
> our Mysql solutions.
>
> Pending that Lassosoft fixes this, any clues on how I can trap for 4 byte
> chars in user submitted data?
> So that it would be possible to either remove/replace them or send back a
> good looking error message asking the user to clean up the data first.
>
> HDB
> Jolle
>
> #############################################################
>
> This message is sent to you because you are subscribed to
>   the mailing list Lasso [hidden email]
> Official list archives available at http://www.lassotalk.com
> To unsubscribe, E-mail to: <[hidden email]>
> Send administrative queries to  <[hidden email]>
>

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
17 feb 2015 kl. 08:53 skrev Bil Corry <[hidden email]>:

> You might be able to use regex, this is untested, but it may work:
>
>     [\\x{10000}-\\x{10ffff}]
>
>
> - Bil

That was an excellent pointer. Thanks Bil!

Here’s working code:
#myinput -> replace(regexp(`[\x{10000}-\x{10ffff}]`, '◊’))

Will turn this:
text 👮 text
to
text ◊ text

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
In reply to this post by Bil Corry-3
17 feb 2015 kl. 08:38 skrev Bil Corry <[hidden email]>:

> You might try sending:
>
>    SET NAMES 'utf8mb4'
>
> at the beginning of the connection with MySQL, then try your queries.  That
> should set the connection to send/receive 4-byte UTF-8 characters.

Will help with something. But not achieve the goal.

This:
myds -> sql("SET NAMES 'utf8mb4';
INSERT INTO mytable SET mytext = 'text 😄 text 😃 text 😊 text 😣 text 👮 text 🏃 text 👣';”)

will happily create the record. But the data stored will look like this:

 text ? text ? text ? text ? text ? text ? text ?

Both when viewed thru Navicat and when retrieved thru Lasso. Remove the SET NAMES 'utf8mb4'; and the request will fail as before.

The regex cleanup you pointed to is the route I am taking until Lassosoft offers a permanent fix.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
In reply to this post by Jolle Carlestam-2
Hm, I just figured out a cool prank. If we start every mail to this list with a smiley face it will render the Lassotalk archive useless.

HDB
Jolle
😄

^^^
|||

See!

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
In reply to this post by Jolle Carlestam-2

17 feb 2015 kl. 18:57 skrev Jolle Carlestam <[hidden email]>:

> 17 feb 2015 kl. 08:53 skrev Bil Corry <[hidden email]>:
>
>> You might be able to use regex, this is untested, but it may work:
>>
>>    [\\x{10000}-\\x{10ffff}]
>>
>>
>> - Bil
>
> That was an excellent pointer. Thanks Bil!
>
> Here’s working code:
> #myinput -> replace(regexp(`[\x{10000}-\x{10ffff}]`, '◊’))
>

And here’s a method to do it for us:

define string -> cleanupemoji(replacechar:: string = '◊') => {
        .replace(regexp(`[\x{10000}-\x{10ffff}]`, #replacechar))
}

Example:
#myinput -> cleanupemoji
or
#myinput -> cleanupemoji(’’)


HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

RE: Emoticon UTF-8 issues when adding rows to Mysql table

Rick Draper-2
Just to add to this discussion, I was asked at a presentation this week if our solution supported Emoji characters in passwords!!

The keyboard is on most mobile devices and it seems that some people are wanting to use it as part of their password string.

VBR

Rick



#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
19 feb 2015 kl. 23:03 skrev Rick Draper <[hidden email]>:

> Just to add to this discussion, I was asked at a presentation this week if our solution supported Emoji characters in passwords!!
>
> The keyboard is on most mobile devices and it seems that some people are wanting to use it as part of their password string.
>
> VBR
>
> Rick

My initial thought was that this is another argument why Lassosoft need to fix this bug ASAP.
Then I realized that you can safely claim that your solution will indeed support Emojis in passwords. Since the passwords are not stored as is anyway. They are always hashed and the hash value is no problem for the DB or connector to handle.

You can, however, not store user names with Emojis in them…

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

RE: Emoticon UTF-8 issues when adding rows to Mysql table

Rick Draper-2
> They are always hashed and the hash value is no problem for the DB or
connector to handle.

Indeed - I wondered at the time if it was an informed question... it sure
seemed to be, and your thread was most timely.

VBR

Rick




#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Jolle Carlestam-2
In reply to this post by Jolle Carlestam-2
Here’s a reminder that this issue is still at large.
I can confirm that it’s still not working in Lasso 9.3.1 beta.

If you don’t remember the issue, Lasso 9 can’t store emojis or other four byte chars in Mysql. All evidence points to the connector. Here’s test code to verify it with:

inline(-Add,
        -Database = 'emojitest',
        -Table = 'emoji',
        -tableEncoding = 'utf8mb4',
        'text' = ’ [replace with your own emoji char here]'
) => {^
error_msg
^}


The reason I don’t supply the actual emoji in the demo code is due to the fact that if I do it will fail to register in Lassotalk.

I have tried with or without the tableEncoding param.

HDB
Jolle

#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Mason Miller
As soon as they fix the problem, all my passwords that contain Emoji's will be toast!



Mason




---- On Wed, 18 Nov 2015 07:34:46 -0800 Jolle Carlestam &lt;[hidden email]&gt;wrote ----




Here’s a reminder that this issue is still at large.

I can confirm that it’s still not working in Lasso 9.3.1 beta.



If you don’t remember the issue, Lasso 9 can’t store emojis or other four byte chars in Mysql. All evidence points to the connector. Here’s test code to verify it with:



inline(-Add,

    -Database        = 'emojitest',

    -Table            = 'emoji',

    -tableEncoding    = 'utf8mb4',

    'text'            = ’ [replace with your own emoji char here]'

) =&gt; {^

error_msg

^}





The reason I don’t supply the actual emoji in the demo code is due to the fact that if I do it will fail to register in Lassotalk.



I have tried with or without the tableEncoding param.



HDB

Jolle



#############################################################



This message is sent to you because you are subscribed to

 the mailing list Lasso [hidden email]

Official list archives available at http://www.lassotalk.com

To unsubscribe, E-mail to: &lt;[hidden email]&gt;

Send administrative queries to &lt;[hidden email]&gt;






#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
Reply | Threaded
Open this post in threaded view
|

Re: Emoticon UTF-8 issues when adding rows to Mysql table

Mason Miller
In reply to this post by Jolle Carlestam-2
If the problem is the Lasso DB connector, would you just pass it to the OS and have python or php connect to the db?



Mason




---- On Wed, 18 Nov 2015 07:34:46 -0800 Jolle Carlestam &lt;[hidden email]&gt;wrote ----




Here’s a reminder that this issue is still at large.

I can confirm that it’s still not working in Lasso 9.3.1 beta.



If you don’t remember the issue, Lasso 9 can’t store emojis or other four byte chars in Mysql. All evidence points to the connector. Here’s test code to verify it with:



inline(-Add,

    -Database        = 'emojitest',

    -Table            = 'emoji',

    -tableEncoding    = 'utf8mb4',

    'text'            = ’ [replace with your own emoji char here]'

) =&gt; {^

error_msg

^}





The reason I don’t supply the actual emoji in the demo code is due to the fact that if I do it will fail to register in Lassotalk.



I have tried with or without the tableEncoding param.



HDB

Jolle



#############################################################



This message is sent to you because you are subscribed to

 the mailing list Lasso [hidden email]

Official list archives available at http://www.lassotalk.com

To unsubscribe, E-mail to: &lt;[hidden email]&gt;

Send administrative queries to &lt;[hidden email]&gt;






#############################################################

This message is sent to you because you are subscribed to
  the mailing list Lasso [hidden email]
Official list archives available at http://www.lassotalk.com
To unsubscribe, E-mail to: <[hidden email]>
Send administrative queries to  <[hidden email]>
12