Tuesday, July 14, 2009

Linux Vim 编码 (转载)

原始网址:
http://xwrabit.blog.163.com/blog/static/25675746200861510285091/

和所有的流行文本编辑器一样,Vim 可以很好的编辑各种字符编码的文件,这当然包括UCS-2、UTF-8 等流行的 Unicode 编码方式。然而不幸的是,和很多来自 Linux 世界的软件一样,这需要你自己动手设置。

Vim 有四个跟字符编码方式有关的选项,encoding、fileencoding、fileencodings、termencoding (这些选项可能的取值请参考 Vim 在线帮助 :help encoding-names),它们的意义如下:

* encoding: Vim 内部使用的字符编码方式,包括 Vim 的 buffer (缓冲区)、菜单文本、消息文本等。默认是根据你的locale选择.用户手册上建议只在 .vimrc 中改变它的值,事实上似乎也只有在.vimrc 中改变它的值才有意义。你可以用另外一种编码来编辑和保存文件,如你的vim的encoding为utf-8,所编辑的文件采用cp936编码,vim会自动将读入的文件转成utf-8(vim的能读懂的方式),而当你写入文件时,又会自动转回成cp936(文件的保存编码).

* fileencoding: Vim 中当前编辑的文件的字符编码方式,Vim 保存文件时也会将文件保存为这种字符编码方式 (不管是否新文件都如此)。

* fileencodings: Vim自动探测fileencoding的顺序列表,启动时会按照它所列出的字符编码方式逐一探测即将打开的文件的字符编码方式,并且将 fileencoding 设置为最终探测到的字符编码方式。因此最好将Unicode 编码方式放到这个列表的最前面,将拉丁语系编码方式 latin1 放到最后面。

* termencoding: Vim 所工作的终端 (或者 Windows 的 Console 窗口) 的字符编码方式。如果vim所在的term与vim编码相同,则无需设置。如其不然,你可以用vim的termencoding选项将自动转换成term 的编码.这个选项在 Windows 下对我们常用的 GUI 模式的 gVim 无效,而对 Console 模式的Vim 而言就是 Windows 控制台的代码页,并且通常我们不需要改变它。

好了,解释完了这一堆容易让新手犯糊涂的参数,我们来看看 Vim 的多字符编码方式支持是如何工作的。

1. Vim 启动,根据 .vimrc 中设置的 encoding 的值来设置 buffer、菜单文本、消息文的字符编码方式。

2. 读取需要编辑的文件,根据 fileencodings 中列出的字符编码方式逐一探测该文件编码方式。并设置 fileencoding 为探测到的,看起来是正确的 (注1) 字符编码方式。

3. 对比 fileencoding 和 encoding 的值,若不同则调用 iconv 将文件内容转换为encoding 所描述的字符编码方式,并且把转换后的内容放到为此文件开辟的 buffer 里,此时我们就可以开始编辑这个文件了。注意,完成这一步动作需要调用外部的 iconv.dll(注2),你需要保证这个文件存在于 $VIMRUNTIME 或者其他列在 PATH 环境变量中的目录里。

4. 编辑完成后保存文件时,再次对比 fileencoding 和 encoding 的值。若不同,再次调用 iconv 将即将保存的 buffer 中的文本转换为 fileencoding 所描述的字符编码方式,并保存到指定的文件中。同样,这需要调用 iconv.dll由于 Unicode 能够包含几乎所有的语言的字符,而且 Unicode 的 UTF-8 编码方式又是非常具有性价比的编码方式 (空间消耗比 UCS-2 小),因此建议 encoding 的值设置为utf-8。这么做的另一个理由是 encoding 设置为 utf-8 时,Vim 自动探测文件的编码方式会更准确 (或许这个理由才是主要的 ;)。我们在中文 Windows 里编辑的文件,为了兼顾与其他软件的兼容性,文件编码还是设置为 GB2312/GBK 比较合适,因此 fileencoding 建议设置为 chinese (chinese 是个别名,在 Unix 里表示 gb2312,在 Windows 里表示cp936,也就是 GBK 的代码页)。

以下是我的 .vimrc(见附件) 中关于字符编码方式设置的内容,这个设置比较有弹性,可以根据系统中的环境变量 $LANG (当然,Windows 中的写法是 %LANG%) 的值来自动设置合适的字符编码方式。此时,推荐设置 %LANG% = zh_CN.UTF-8,可以通过后面的 Windows 注册表脚本文件来方便的做到。

注1: 事实上,Vim 的探测准确度并不高,尤其是在 encoding 没有设置为 utf-8 时。因此强烈建议将 encoding 设置为 utf-8,虽然如果你想 Vim 显示中文菜单和提示消息的话这样会带来另一个小问题。

注2: 在 GNU 的 FTP 上可以下载到 iconv 的 Win32 版(http://mirrors.kernel.org/gnu/libiconv/libiconv-1.9.1.bin.woe32.zip),不推荐去GnuWin32(http://gnuwin32.sourceforge.net/) 下载 libiconv,因为那个版本旧一些,并且需要自己改名 dll 文件。

注3: 查看帮助 :h iconv-dynamic

On MS-Windows Vim can be compiled with the |+iconv/dyn| feature. This means Vim will search for the "iconv.dll" and "libiconv.dll" libraries. When neither of them can be found Vim will still work but some conversions won't be possible.

附1:vimrc文件

" Multi-encoding setting, MUST BE IN THE BEGINNING OF .vimrc!

"

if has("multi_byte")

" When 'fileencodings' starts with 'ucs-bom', don't do this manually

"set bomb

set fileencodings=ucs-bom,chinese,taiwan,japan,korea,utf-8,latin1

" CJK environment detection and corresponding setting

if v:lang =~ "^zh_CN"

" Simplified Chinese, on Unix euc-cn, on MS-Windows cp936

set encoding=chinese

set termencoding=chinese

if &fileencoding == ''

set fileencoding=chinese

endif

elseif v:lang =~ "^zh_TW"

" Traditional Chinese, on Unix euc-tw, on MS-Windows cp950

set encoding=taiwan

set termencoding=taiwan

if &fileencoding == ''

set fileencoding=taiwan

endif

elseif v:lang =~ "^ja_JP"

" Japanese, on Unix euc-jp, on MS-Windows cp932

set encoding=japan

set termencoding=japan

if &fileencoding == ''

set fileencoding=japan

endif

elseif v:lang =~ "^ko"

" Korean on Unix euc-kr, on MS-Windows cp949

set encoding=korea

set termencoding=korea

if &fileencoding == ''

set fileencoding=korea

endif

endif

" Detect UTF-8 locale, and override CJK setting if needed

if v:lang =~ "utf8$" || v:lang =~ "UTF-8$"

set encoding=utf-8

endif

else

echoerr 'Sorry, this version of (g)Vim was not compiled with "multi_byte"'

endif

附2:

Supported 'encoding' values are: *encoding-values*

1 latin1 8-bit characters (ISO 8859-1)

1 iso-8859-n ISO_8859 variant (n = 2 to 15)

1 koi8-r Russian

1 koi8-u Ukrainian

1 macroman MacRoman (Macintosh encoding)

1 8bit-{name} any 8-bit encoding (Vim specific name)

1 cp437 similar to iso-8859-1

1 cp737 similar to iso-8859-7

1 cp775 Baltic

1 cp850 similar to iso-8859-4

1 cp852 similar to iso-8859-1

1 cp855 similar to iso-8859-2

1 cp857 similar to iso-8859-5

1 cp860 similar to iso-8859-9

1 cp861 similar to iso-8859-1

1 cp862 similar to iso-8859-1

1 cp863 similar to iso-8859-8

1 cp865 similar to iso-8859-1

1 cp866 similar to iso-8859-5

1 cp869 similar to iso-8859-7

1 cp874 Thai

1 cp1250 Czech, Polish, etc.

1 cp1251 Cyrillic

1 cp1253 Greek

1 cp1254 Turkish

1 cp1255 Hebrew

1 cp1256 Arabic

1 cp1257 Baltic

1 cp1258 Vietnamese

1 cp{number} MS-Windows: any installed single-byte codepage

2 cp932 Japanese (Windows only)

2 euc-jp Japanese (Unix only)

2 sjis Japanese (Unix only)

2 cp949 Korean (Unix and Windows)

2 euc-kr Korean (Unix only)

2 cp936 simplified Chinese (Windows only)

2 euc-cn simplified Chinese (Unix only)

2 cp950 traditional Chinese (on Unix alias for big5)

2 big5 traditional Chinese (on Windows alias for cp950)

2 euc-tw traditional Chinese (Unix only)

2 2byte-{name} Unix: any double-byte encoding (Vim specific name)

2 cp{number} MS-Windows: any installed double-byte codepage

u utf-8 32 bit UTF-8 encoded Unicode (ISO/IEC 10646-1)

u ucs-2 16 bit UCS-2 encoded Unicode (ISO/IEC 10646-1)

u ucs-2le like ucs-2, little endian

u utf-16 ucs-2 extended with double-words for more characters

u utf-16le like utf-16, little endian

u ucs-4 32 bit UCS-4 encoded Unicode (ISO/IEC 10646-1)

u ucs-4le like ucs-4, little endian

The {name} can be any encoding name that your system supports. It is passed

to iconv() to convert between the encoding of the file and the current locale.

For MS-Windows "cp{number}" means using codepage {number}.

Several aliases can be used, they are translated to one of the names above.

An incomplete list:

1 ansi same as latin1 (obsolete, for backward compatibility)

2 japan Japanese: on Unix "euc-jp", on MS-Windows cp932

2 korea Korean: on Unix "euc-kr", on MS-Windows cp949

2 prc simplified Chinese: on Unix "euc-cn", on MS-Windows cp936

2 chinese same as "prc"

2 taiwan traditional Chinese: on Unix "euc-tw", on MS-Windows cp950

u utf8 same as utf-8

u unicode same as ucs-2

u ucs2be same as ucs-2 (big endian)

u ucs-2be same as ucs-2 (big endian)

u ucs-4be same as ucs-4 (big endian)

default stands for the default value of 'encoding', depends on the

environment

Saturday, July 11, 2009

First step of Ubuntu

0. useful tool to find packages of ubuntu
sudo aptitude


1. install some programming language and packages

for C++:
aptitude install g++
aptitude install build-essential
sudo apt-get install zlib1g-dev libssl-dev (for irstlm)

for Java:
sudo apt-get install sun-java6-jdk
and set environment variable
export JAVA_HOME="/usr/lib/jvm/java-6-sun;"
in ~/.bash_profile


2. install flash plugin for firefox
download flash player (.tar.gz for Linux version) from:
http://get.adobe.com/flashplayer/
decompression it, and copy the libflashplayer.so to /usr/lib/firefox-addons/



3. install Flock browser
sudo apt-get install libstdc++5
download Flock from:
http://www.flock.com/download/versions
tar xvf flock-2.5.en-US.linux-i686.tar.bz2
./flock-browser
install flash plugin to Flock:
just like the case of Firefox, copy the libflashplayer.so of flash player to flock/plugins/



4. instal Opera browser
download Opera 10.00b1 for Linux i386 (Download this package in TAR.GZ format) from:
http://www.opera.com/download/?os=linux-i386&ver=10.00b1&local=y
tar -xvzf opera-10.00-b1.gcc4-shared-qt3.i386.tar.gz
cd opera-10.00-4402.gcc4-shared-qt3.i386/

sudo ./install.sh
And then install flash plugin like the firefox case:
copy the libflashplayer.so to /usr/lib/opera/plugins

Tuesday, July 7, 2009

In Java, how to get the OS type and OS name

String osName=null;
// get OS type
try
{
String osType = System.getProperty("os.arch");
System.out.println("Operating system type => " + osType);
osName = System.getProperty("os.name");
System.out.println("Operating system type => " + osName);
}
catch (Exception e)
{
System.out.println("Exception caught =" + e.getMessage());
}

Set Netbeans user Interface language

Netbeans automatically uses the Windows system default language as the default user interface language. I believe that it means to be a nice feature for localization. But I personally find it uncomfortable because I have been used with English interface.

After I did some Google search, I learned a few tips to set the Netbeans UI language.

1. Temporary Solution

Add "--locale en:US" at the end of Netbeans startup command.

"C:\Program Files\NetBeans 6.0.1\bin\netbeans.exe" --locale en:US

2. Permanent Solution

Go to Netbeans installation directory, for example,

C:\Program Files\NetBeans 6.0.1\etc

Open "netbeans.conf" and find netbeans default option line

netbeans_default_options="-J-client -J-Xss2m -J-Xms32m -J-XX:PermSize=32m -J-XX:MaxPermSize=200m -J-Xverify:none -J-Dapple.laf.useScreenMenuBar=true"

Add "-J-Duser.language=en -J-Duser.region=US" to the end of this line

netbeans_default_options="-J-client -J-Xss2m -J-Xms32m -J-XX:PermSize=32m -J-XX:MaxPermSize=200m -J-Xverify:none -J-Dapple.laf.useScreenMenuBar=true -J-Duser.language=en -J-Duser.region=US"


It would be nice that there is an option to allow me choose the user interface language in the next Netbeans release.

In python, how to read unicode or GBK files

import codecs

infile=codecs.open(infilename,"r","gbk")

line=infile.readline()

# all the lines which are read from infile are encoded in GBK

Saturday, July 4, 2009

In Java, how do I set a default character encoding for file I/O operations

The default encoding used by locale/encoding sensitive API in the Java libraries is determined by the System property "file.encoding". This system property is initialized by the JVM startup code after querying the underlying native operating system. For example on my English USA NT box it is initialized to:

Cp1252

It is generally recommended that you do not modify it. However if you know what you are doing you could override the system property either on the command line using the -

java -Dfile.encoding=GBK -jar Speech.jar

syntax or programmatically at startup.

Friday, July 3, 2009

Adding a DOS cmd prompt to the Windows right click menu

1.
Go to the HKEY_CLASSES_ROOT\Folder\shell node in the registry (note that Folder includes drives and directories).

2.
Create a new key (from the right-click menu), and call it "DOS prompt".

3.
Under that new key, create a sub-key named "command".

4.
Under that new key (i.e under HKEY_CLASSES_ROOT\Folder\shell\DOS prompt\command), edit the (Default) String value and give it the following value: "c:\windows\system32\cmd.exe %1" (or whatever the path to cmd.exe is on your system).

5.
Now, you can open a DOS window by right-clicking on any directory name in the Windows explorer.

Wednesday, July 1, 2009

In Java, how to change the encoding of files

These days, I have a task to change the file whose encoding is UTF-8 into a file with encoding of GBK.

I found a very useful tutorial at:
http://www.herongyang.com/unicode/jdk_encoding_conversion.html

As a summary, you can directly use the following Java code:

String cfilename="";
String outCnFileName="";

BufferedReader cfile = new BufferedReader( new InputStreamReader(new FileInputStream(cfilename),"UTF-8") );
BufferedWriter outcnfile = new BufferedWriter( new OutputStreamWriter(new FileOutputStream(outCnFileName),"GBK") );

String line=cfile.readLine();
while( line!=null )
{
outcnfile.write(line+"\n");
}

cfile.close();
outcnfile.close();