+++ /dev/null
-From pgsql-patches-owner+M16987@postgresql.org Thu Aug 4 17:35:52 2005
-Return-path: <pgsql-patches-owner+M16987@postgresql.org>
-Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71])
- by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j74LZot06577
- for <pgman@candle.pha.pa.us>; Thu, 4 Aug 2005 17:35:50 -0400 (EDT)
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id ED433529DB;
- Thu, 4 Aug 2005 18:35:44 -0300 (ADT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 70225-08; Thu, 4 Aug 2005 21:35:44 +0000 (GMT)
-Received: from postgresql.org (svr1.postgresql.org [200.46.204.71])
- by svr1.postgresql.org (Postfix) with ESMTP id 0ECB452998;
- Thu, 4 Aug 2005 18:35:44 -0300 (ADT)
-X-Original-To: pgsql-patches-postgresql.org@localhost.postgresql.org
-Received: from localhost (unknown [200.46.204.144])
- by svr1.postgresql.org (Postfix) with ESMTP id AFAF752840
- for <pgsql-patches-postgresql.org@localhost.postgresql.org>; Thu, 4 Aug 2005 18:33:25 -0300 (ADT)
-Received: from svr1.postgresql.org ([200.46.204.71])
- by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024)
- with ESMTP id 70920-01
- for <pgsql-patches-postgresql.org@localhost.postgresql.org>;
- Thu, 4 Aug 2005 21:33:13 +0000 (GMT)
-Received: from mx-2.sollentuna.net (mx-2.sollentuna.net [195.84.163.199])
- by svr1.postgresql.org (Postfix) with ESMTP id E1C935280D
- for <pgsql-patches@postgresql.org>; Thu, 4 Aug 2005 18:33:11 -0300 (ADT)
-Received: from ALGOL.sollentuna.se (janus.sollentuna.se [62.65.68.67])
- by mx-2.sollentuna.net (Postfix) with ESMTP id 7F5A08F289
- for <pgsql-patches@postgresql.org>; Thu, 4 Aug 2005 23:33:13 +0200 (CEST)
-content-class: urn:content-classes:message
-MIME-Version: 1.0
-Content-Type: multipart/mixed;
- boundary="----_=_NextPart_001_01C5993C.1E1CB100"
-Subject: [PATCHES] FW: Win32 unicode vs ICU
-X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0
-Date: Thu, 4 Aug 2005 23:33:12 +0200
-Message-ID: <6BCB9D8A16AC4241919521715F4D8BCE094656@algol.sollentuna.se>
-X-MS-Has-Attach: yes
-Thread-Topic: Win32 unicode vs ICU
-Thread-Index: AcWVyKf6M9GrCqZAQGeSvCZNg9n/jgDc1WwQ
-From: "Magnus Hagander" <mha@sollentuna.net>
-To: "PostgreSQL-patches" <pgsql-patches@postgresql.org>
-X-Virus-Scanned: by amavisd-new at hub.org
-X-Mailing-List: pgsql-patches
-List-Archive: <http://archives.postgresql.org/pgsql-patches>
-List-Help: <mailto:majordomo@postgresql.org?body=help>
-List-Id: <pgsql-patches.postgresql.org>
-List-Owner: <mailto:pgsql-patches-owner@postgresql.org>
-List-Post: <mailto:pgsql-patches@postgresql.org>
-List-Subscribe: <mailto:majordomo@postgresql.org?body=sub%20pgsql-patches>
-List-Unsubscribe: <mailto:majordomo@postgresql.org?body=unsub%20pgsql-patches>
-Precedence: bulk
-Sender: pgsql-patches-owner@postgresql.org
-X-Virus-Scanned: by amavisd-new at hub.org
-Status: ORr
-
-This is a multi-part message in MIME format.
-
-------_=_NextPart_001_01C5993C.1E1CB100
-Content-Type: text/plain;
- charset="us-ascii"
-Content-Transfer-Encoding: quoted-printable
-
-I just realised this mail didn't go through. Probably because it was too
-large for -hackers. So: repost to -patches. Sorry about that. If it's a
-duplicate, even more sorry, but I couldn't find it in the archives.
-
-(This may explain that nobody answered me :P)
-
-//Magnus
-=20
-
-> -----Original Message-----
-> From: Magnus Hagander=20
-> Sent: Sunday, July 31, 2005 2:09 PM
-> To: PostgreSQL-development
-> Cc: pgsql-hackers-win32@postgresql.org
-> Subject: Win32 unicode vs ICU
->=20
-> Hi!
->=20
-> I've been working with Palles ICU patch to make it work on=20
-> win32, and I believe I have it done. While doing it I noticed=20
-> that ICU basically converts to UTF16 and back - I previously=20
-> thought it worked on UTF8 strings. Based on this I also tried=20
-> out an implementation for the win32-unicode problem that does=20
-> *not* require ICU. It uses the win32 native functions to map=20
-> to utf16 and back, and then to process the text there. And I=20
-> got through with much less code than the ICU version, while=20
-> doing the same thing.
->=20
-> I am unsure of how to proceed. As I see it there are three paths:
-> 1) Use native win32 functionality only on win32
-> 2) Use ICU functionality only on win32
-> 3) Allow both ICU and native functionality, compile time=20
-> switch --with-icu (same as unix with the ICU patch)
->=20
->=20
-> The main downsides of ICU vs the native ones are:
-> * ICU does not accept win32 locale names. When doing=20
-> setlocale("sv_se"), for example, win32 will return this in=20
-> later calls as "Swedish_Sweden.1252". To get around this in=20
-> the ICU patch, I had to implement a lookup map that converts=20
-> it back to sv_se for ICU.
->=20
-> * ICU is yet another build and runtime dependency, and a=20
-> large one (comes in at 11Mb for the DLL files alone in the=20
-> win32 download)
->=20
->=20
-> I guess that the main upside of it is that we'd get=20
-> constistent behaviour - in case there are issues with either=20
-> ICU or win32 native they'd otherwise differ. And only one new=20
-> codepath. But we already live with the platform-inconsistency today...
->=20
-> Another upside is that it handles more encodings in ICU - my=20
-> native implementation does *only* UTF8 and relies on existing=20
-> functionality to deal with other encodings. It could of=20
-> course be extended if necessary, but from what I can tell=20
-> UTF8 is the big one.
->=20
->=20
->=20
-> I have attached both patches. For the native version, only=20
-> win32_utf8.patch is required. For the ICU version,=20
-> icu_win32.patch is needed and also the files=20
-> localemap.c,localemap.pl, iso639 and iso3166 needs to go in=20
-> src/backend/port/win32. (the localemap needs to be updated to=20
-> do a better-than-linear search, but I wanted to include an example)
->=20
->=20
-> Thoughts on the options?
->=20
->=20
-> And anohter question - my native patch touches the same=20
-> functions as the ICU patch. Can somebody who knows the=20
-> internals confirm or deny that these are all the required=20
-> locations, or do we need to modify more?
->=20
-> (I have run simple tests in swedish locale and both behave=20
-> the same and correct, but I'm unsure of exactly how much=20
-> would be affected)
->=20
-> Finally, the win32 patch also changes the normal path to use=20
-> strncoll(). The comment above the function states that we'd=20
-> like to use strncoll but it's not available. Well, on win32=20
-> it is, so it should provide a speedup on win32. It is=20
-> currently not included in the ICU patch, but should probably=20
-> be included whichever path we'd chose.
->=20
->=20
->&nb
projects / users / gsingh / postgres.git / commitdiff