my first stop would be the W3C mobileOK Checker:
http://validator.w3.org/mobile/ [validator.w3.org]

Interesting check, trouble is that I don't know how much to trust when it reported a pop-up which certainly doesn't exist.

Thanks Pranque,
Doing some changes, did not get that bad result,
41 points = acceptable but not perfect.
no extrem error, 2 several,
changing iso to utf-8 as I had it for the latin characters.
when doing some changes will post again.
Helena

Having problems in changing my charset=iso-8859-1 to charset=UTF-8.

It works perfect on plain text if I write in design mode instead of code mode, then dreamweaver autmatically writes the html version of the special characters.

The problem is with the forms, read I have to put utf-8 in header and as well in form.
This I did, however when I write cañon guía, in the email I get cañon guía.
So it does not work with the form, dont know why, and on top I have information that I get from a database that are in latin1_swedish_ci, so the ñ I get as a figure with a ? inside, maybe the database dont have to bee changed to utf-8, maybe some string replace can do it?
However I does not work to fill in latin characters in the form.
Please help, thanks in advance.
Helen

Uups, this looks really complicated, dont know if certain, but all files included nead to be coded in utf-8,
and there are many php files included. In dreamweaver I cant choose and dont have the default as utf-8.
So I opened some with notepad, saved them as utf-8, when I upploaded on the pages that where not utf-8 I got some strange signs....
And is it necesarry to change the database also? Please advice.

It's a horrible solution, but if you express all non-ASCII chracters as html entities a word my fingers flatly refuse to type right on the first try then the stated file encoding doesn't matter.

This I did, however when I write cañon guía, in the email I get cañon guía.

Yup. What you've got there is UTF-8 encoded text being reinterpreted as ISO-Latin-1.

What language is the database in? Not what human language ;) I mean what computer language? There's almost always some type of encode/decode function to deal with non-ASCII characters, or characters that need to be escaped for other reasons.

And remember that changing the html header's "charset" does not change the actual file, it only changes how the browser reads it. So it has to be composed in-- or changed to-- UTF-8 in the first place.

:: happily patting SubEthaEdit on the head because it will both convert and reinterpret on the fly ::

Dear Lucy,
Thanks,
As I said the database is in latin1_swedish_ci and handle the characters well.
Just tried a program to convert the files to utf-8 and did not work,
How do I change it to utf-8 except changing the charset?

I dont have any problem with writing the html for the swedish and latin characters, as writing in design mode, dreamweaver does it for you.

you can use the GNU iconv utility to convert to a different encoding.
there is also a PHP module that provides iconv support.

you might find this thread informative - Character encoding, entity references and UTF-8:
http://www.webmasterworld.com/forum21/11176.htm [webmasterworld.com]

As I said the database is in latin1_swedish_ci and handle the characters well.

Is that a programming language? It sounds like a file encoding.

Just tried a program to convert the files to utf-8 and did not work,

What did it do? That is, in what way did it not work, and how could you tell?

How do I change it to utf-8 except changing the charset?

The "charset" declaration in an html file does not change anything. The only thing it does is tell the user's browser how to interpret any given byte or group of bytes.

You know of course that what travels across the Internet is not abcde... It is just a string of 01001100 et cetera. And then the device at the far end takes those 0's and 1's, puts them into sets of eight, and changes them into displayed characters. The same thing happens at the "near" end when you are typing your text. (Note also that it makes no difference what you did to make the character in the first place: "dead keys", alt + a string of letters, a Swedish keyboard, et cetera. That's another and completely unrelated issue.)

when I write cañon guía, in the email I get cañon guía.

Long version: ñ in UTF-8 is four bytes, C3 B1. But the device at the other end doesn't know this. If it thinks the text is supposed to be Latin-1, then your ñ becomes two separate characters, C3 and B1, giving you ñ. (The first letter of the pair will always be à because this whole block starts with C3. Another block I'm very familiar with, the UCAS range, always turns into á followed by two random letters, because the letters are three sets of two bytes, starting with E1.)

So it does not work with the form, dont know why, and on top I have information that I get from a database that are in latin1_swedish_ci, so the ñ I get as a figure with a ? inside

That question mark is the UTF-8 "I can't deal with this" replacement character. You get it when your original text was in Latin-1 (or other 1-byte encodings including Mac Roman), which uses codepoints that UTF-8 doesn't use. That includes your ñ, along with letters like ä ö and å that must crop up sooner or later ;)

Is she ever going to get to the point?

Uhm. The point is that if the HTML "charset" declaration says UTF-8, then you have to make your original raw text into UTF-8. Exactly how you do this will depend on your text editor or html editor. There might be a popup or menu item somewhere, or something you change in the Preferences. In extreme cases you might even have to read the manual.

everything displayed on your page must be in the same encoding - text from the database, templates, included files, etc.

then your html document should have the encoding declaration immediately after the document head, like:
...

regarding your database, you will likely have to dump the database, convert the file to utf8, change all the latin1 declarations to utf8, create a new database with utf8 character support, and import the converted db dump file.

Thanks prnanque,
am trying the gnu iconv, however dont understand, I have windows,
anyway, with this:
everything displayed on your page must be in the same encoding - text from the database, templates, included files, etc.

Do you mean the page must be saved as utf-8 (thing I cant see I can do with dreamweaver, I can only change the default setting for new documents).
Or do you mean it must be same encoding, if its not I will see as I will get an error, so I could just change where I see the characters are wrong? most of the texts are already done in html version.

Most php files that are included dont have any meta or anything just the php code.

[edited by: helenp at 9:14 am (utc) on Jul 29, 2011]

Thanks Lucy,
Dont know what you mean with language in the database,
I use mysql.

I used a program, converted the whole folder (a copy of course) and in the pc, at this moment I dont have php installed, so the php part I cant preview, and I opened a normal text page and everything was upside down, no photos, color from css was wrong, a desaster.
Thats why I said it did not work.

Converted everything except database and links in some files (all includes converted, and still dont work when filling in a form.

To test, I did a small cgi form the host offer,
this is the code of the page, no includes, no databases nothing:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>

<body>
<FORM ACTION="http://mydomain.com/cgi-sys/FormMail.cgi" METHOD="POST" accept-charset="UTF-8">
<input type=hidden name="recipient" value="info@mydomain.com">
<input type=hidden name="subject" value="Asunto del mail que se envia">

<table border="0" cellspacing="2" cellpadding="2">
<tr>
<td><font face="Verdana, Arial, Helvetica, sans-serif" size="2">Nombre:<br>

<input type="text" name="nombre" size="25" maxlength="300">
</font></td>
<td> <font face="Verdana, Arial, Helvetica, sans-serif" size="2">Email:<br>
<input type="text" name="email" size="25" maxlength="300">
</font></td>
</tr>
<tr>
<td colspan="2">

<div align="center"><br>
<input type="submit" value="Enviar">
</td>
</tr>
</table>
</form>
</body>
</html>

This I wrote in the form: cañon guía åäö
This was the result in the cgi page:
Form Submission Results
nombre: cañon guía åäö
Everything looks ok.
However I receive the email like this:
nombre: cañon guía åäö

Whats wrong?
Please and thanks.

Just dont get it,
used the same form on chrome and ie,
and got bad caracters in cgi pages also....
Chrome and IE
Form Submission Results

nombre: `ñññññ

Can it be a problem witht the host?

When you say "the e-mail" do you mean your personal e-mail reader, or a preview of the mail that will be sent? The change from
cañon guía åäö
to
cañon guía åäö
is a single error that happens just once. So either the e-mail isn't sending the right encoding information, or your e-mail reader isn't seeing it.

If you're talking about e-mail that has actually been sent, look at the "content-type" in the e-mail header; it should give an encoding. (Frankly I have no idea where it gets the information. Mine apparently defaults to Latin-1, but silently jumps to UTF-8 when needed.) Then at least you'll know which end is making the mistake.

Dear Lucy,
Thanks,
I was talking about the email that has been sent.
I looked at the header and there were no content-type at all in it. I sent a new one, this time I did not download it, checked it at cpanels emailprogram, and the same result cañon guía åäö.

Yesterday I also tried html forms, those that you fill in the information in the website, then when you click on enter, outloook (or whatever one have) opens to send the email, same result there, the characters were bad.

I have entered [validator.w3.org...] and checked the test page, and it says encoding utf-8

Just dont understand anything.

However just to check, sent an email from my real form where I changed everything to utf-8 except the database and the content type in the header was:
Content-Type: text/plain; charset="iso-8859-1"
However W3 validated the page as utf-8
dont know if they just read the charset or the encoding.

To change to utf-8 I used PsPad, as one recomended it, just dont understand the libiconv and what command I have to write, and if I have to write every page, or can do the whole folder.
Anyway the testform I did I used dreamweaver with the default set to utf-8 and did a new page, so it should be encoded correctly in utf-8.
Helen

Lol,-
Is it posible to delete some previous post?
Found the reason, why it did not work on the testform no idea, but I do know why it did not work on my form...how stupid, just did not check. I use phpmailer, and in the file class.phpmailer.php it said var $CharSet = 'iso-8859-1'; changed it to var $CharSet = 'utf-8'; and works like a charm lol....
I can change this, is this value the best? var $Encoding = '8bit';
However my formpage got the error in W3:
The document is not valid UTF-8......
just dont get it.

Regarding the database I added mysql_set_charset('utf8'); and I get the characters well, so I dont know if I have to change the database or not.
And if not, is there any reason to change it?

The document is not valid UTF-8

All it takes is one character to sneak in somewhere. Does it show you the character (probably looking like normal text wrapped around The Dread Question Mark), or did the validator go into one of those moods where it flatly refuses to continue?

If the information in the database is now being processed correctly, I don't see any reason to mess with it :)

And then there's the BOM issue...

Dear Lucy,
Dont know, have not checked much the page, will do.
As I see so many errors, did a short text page, just the headers and body etc, just to see result of test, see below.

I dont put the bom, I choosed utf-8 normalizacion c, anyway thats what I read was recomended.

I dont know whats wrong, my code, the w3c test or the host.
I uploaded this small page I just did, only a short text in it:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
Hi this is a test
</body>
</html>

And the errors I get in W3C mobile ok ckecker are:

1. "The document is served without caching information ("Expires" or "Cache-Control" header)"
Logical, there arent any.

2. "The HTTP Content-Type header does not specify a character encoding and no UTF-8 encoding or a non-UTF-8 is specified in the XML declaration."
This I dont understand, there is a character encoding specified as far as I can see......

3. "The document does not validate against XHTML Basic 1.1 or MP 1.2."
Logical, its html, suppose xhtml is important for mobile devices, no idea.

4. "An HTTP response header that matches the meta http-equiv element exists but their values differ
Where?
Triggered by the resource under test:
HTTP header content-type"
Dont understand this either....

5. "The document is not served as "application/xhtml+xml""
Also true, important? no idea

6. " The document uses a non-XML doctype (-//W3C//DTD HTML 4.01 Transitional//EN [w3.org...]
Also true.

All this dont know whats important, however number 2 and 4, I think is correct and there is some bug, or I dont get it?

Don't you wish you could just say the validator is delirious?

It's true that they are obsessive about xhtml instead of html for mobile devices. But I thought that had more to do with e-book readers, not mobiles in general.

Why not change the headers to xhtml and then see what the validator continues to fuss about? Since you're using a "dummy" document, all you have to change in the body is put the text inside <p>tags</p>.

I just tried the same validator on a document of my own that I put online for the specific purpose of being able to access it with my iPad. It works beautifully, even though it is in HTML 4, has background images (colored marginal stripes like on the endpapers of books), uses javascript popups (that's why I couldn't simply make a PDF), has an embedded font* accessed via an at-rule, is much larger than 10K even when you overlook (as the validator did!) the 100K data file and the 200K font... and IT DOES TOO have a charset declaration no matter how many separate times they claim it doesn't ;)

Yeah. Let's say the validator is delirious.


* Because you can't lawfully install fonts in the iPad and it doesn't come with anything that covers the UCAS range.

Lol,
This is the code:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Untitled Document</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
</head>
<body>
<p>Hi this is a test</p>
</body>
</html>

Errors:

1. " The document is served without caching information ("Expires" or "Cache-Control" header)"
Correct

2." The HTTP Content-Type header does not specify a character encoding and no UTF-8 encoding or a non-UTF-8 is specified in the XML declaration"
?

3." An HTTP response header that matches the meta http-equiv element exists but their values differ"
Header content again.
?

4." The document is not served as "application/xhtml+xml""

5." The document uses an XHTML doctype that is not a well-known mobile-friendly doctype (-//W3C//DTD XHTML 1.0 Strict//EN [w3.org...]
lol, the doctype I took from there link.....

This thing fetched a line that had a Server-Side Include and then
complained about what it saw!
Huh?
I scrolled down to the "Source listing" and looked.
Sure enough -- there's the SSI: <!--#include file="license.php" -->
How did the SSI get exposed?