Results tagged “hacking” from muse

这篇文章旨在提供一些编译X窗口系统的指导步骤。细节步骤会因为大家的系统环境不一致而有所不同,而且编译步骤和环境的不同也会造成各种问题。

为什么要编译X窗口系统?你的显卡驱动要求更新版本的X窗口系统,但是 yum 没有更新的版本,所以只能自己编译。

X窗口系统的编译指南 提供了两种编译方法,一种是使用 build.sh 脚本,另一种是使用 jhbuild。这篇文章使用的是 jhbuild,X的官网上也给了 jhbuild 的 编译指南 ,但是比较粗略,这篇文章会详细介绍 jhbuild 的使用,编译X系统时可能遇到的问题,安装和使用X系统时可能遇到的问题。

首先你要安装编译工具和依赖库。这里列出的依赖库可能不完全,请依照实际需要安装更多的软件包。

yum install @gnome-devel @development-tools gnome-common glib2-devel gnome-doc-utils docbook-style-xsl waf
yum install zlib-devel freetype-devel libxslt-devel libxml2-devel expat-devel gperf libgcrypt-devel

安装 python 2.7, pip, 然后用 pip 安装 mako, lxml, six。编译 mesa 时会用到 mako。编译 fontconfig 时会用 lxml 和 six。

yum install centos-release-SCL
scl enable python27 bash
cd /opt/rh/python27/root/usr/bin/ # 进入python2.7安装路径
./easy_install-2.7 pip
./pip2.7 install requests
pip install mako lxml six

准备工作完成后,下载 jhbuildh 开始配置工作。

注意:jhbuild 不能以 root 身份运行,所以你要创建一个新的普通用户并给予 sudoer 权限。

执行 scl enable python27 bash 开启一个新的 shell,在这个 shell 里面,python 是 python 2.7。

以下操作都是以普通用户去执行。

cd # 进入普通用户的 home 目录
git clone git://git.gnome.org/jhbuild
cd jhbuild
./autogen.sh
make
make install

jhbuild 会安装到 ~/.local/bin 目录下,编辑 ~/.bashrc 把这个路径添加到 $PATH 环境变量,用 export PATH=$PATH:$HOME/.local/bin 。然后执行 . ~/.bashrc 去重新加载 .bashrc。

现在你可以找到 jhbuild 这个命令了,用 which jhbuild 去确认一下。

下载使用 jhbuild 去编译 X 的配置文件:

cd # 进入你的 home 目录
mkdir -p xorg/util
git clone git://anongit.freedesktop.org/git/xorg/util/modular/ xorg/util/modular

拷贝 jhbuildrc 文件到你的 home 目录下进行编辑:

cd # 进入你的 home 目录
cp xorg/util/modular/jhbuildrc ./jhbuildrc
nano jhbuildrc

jhbuildrc 是 jhbuild 的配置文件,这个文件实际上是一个 python 脚本,其中你可以配置需要编译哪些 module,编译时的环境变量,编译时配置用的参数,编译和安装路径前缀,等等。

你可以使用 X 提供的默认 jhbuildrc,它会编译整个 X 窗口系统,大概有255个 module。编译路径前缀和安装路径将是你的 home 目录下的 xorg-build。

每个 module 的定义在 xorg/util/modular/xorg.modules 文件里,这个文件实际上是一个XML文档,里面定义了如何去获取源代码,git 仓库的链接在哪里,module 依赖关系。你可以通过修改这个文件去定义某个 module 的版本,比如,你想用11.1.2版本的 mesa,那么你要先查一下 git 仓库里的标签,看哪个 tag 对应的是11.1.2,比如"mesa-11.1.2"对应11.1.2版,那么你在名为 branch 的XML元素里加上 tag="mesa-11.1.2" 即可。(用 git tag -l 去列出 git 标签。)

友情提示:CentOS 6.x 使用的是 HAL,所以你在配置 xserver 时,要用 --enable-config-hal --disable-config-udev ,并且要给编译出来的 Xorg 可执行文件添加 SUID,即 chmod +s Xorg

配置完后 jhbuildrc 文件后,开始编译:

jhbuild -f jhbuildrc

第一次编译时,jhbuild 需要下载 git 仓库,如果你的网络速度不太理想,可能会需要很久。

下面列出来常用的 jhbuild 命令,方便在改变了配置后重新编译。

重新编译所有 module,禁止网络即不更新git信息:

jhbuild -f jhbuildrc build --no-network --force

重新编译一个 module 即 xserver,编译前运行 make clean:

jhbuild -f jhbuildrc buildone xserver --force --clean

在编译时,你可能会遇到各种问题而无法编译下去,jhbuild 会自动问你如何处理编译错误。

Tesseract OCR 中文识别尝试

|

一直以来都想尝试一下中文识别,直到最近才有点空闲时间,主要目的是证实一下到底可不可行,正确率能否达到 95% 以上。自己从头写起十分费时间,因为图片处理很麻烦,所以我选用了 Tesseract OCR。

读者在继续阅读前请注意这篇文章的局限性,在这里我只针对一个楷体字体进行识别,目的是测试一下中文OCR的可行度,以及测试一下 Tesseract OCR。

安装还是在苹果系统上,Mac OS X 10.8.3。Tesseract OCR 版本是目前最新的 v3.02,除了刚开始编译时遇到了点麻烦,其它都很顺利。

因为我是打算识别楷体或行书字体,Tesseract 网站上提供的简体中文数据不太适用,识别率极低,所以便花了些时间自己训练。最后选择是识别楷书,字体是迷你简启体,之所以选这个是因为我当初在找迷你繁智草的顺便收集了。如果以后有空,也可能会尝试识别草书,这个会很有挑战。

训练的过程有些繁琐,主要是识别率太低,需要逐个字检查然后纠正,不过可以写一个简单的Javascript小程序来帮助修改。总体而言步骤还算简单。同时我还发现 Tesseract 的算法对行距很敏感,如果行距太小,根本不识别。(后来我发现,如果把运行参数 -psm 设置成6,Tesseract 还是可以识别的,虽然准确率会降低)另外,字与字之间的间距太小时也经常错误,很显然,英文里的连字(ligature)不能适用于中文。由此可见 Tesseract 其实不太适用于方块字。下面是训练时用到的一个图片:

tesseract-chinese-training-img.png

图片是用 Photoshop 生成的,每个字大概占50x50像素,DPI是72,字体大小是48点。这个设置是针对识别屏幕抓图。

训练数据总共是3508字符,包括标点符号,基本上就是那些通用字。最后生成的数据是20多兆,识别时的运行速度还算快,一百字大概就一两秒,我的CPU是2.6 GHz Intel Core i7。

由于行距是个大问题,我不得不加一个图片处理步骤,就是增加行距。幸好中文是方块字,增加行距十分简单,不需要考虑上下行交叠的问题。图片处理用 PHP 就可以。

下图是从网页直接抓取的图片:

raw-input.png

经过图片处理后的是这样:

ocr-processed-input.png

由于我用的是视网膜屏幕,所以图片有些失真,都是系统直接放大过的。下面是识别出来的文字:

tesseract-chinese-sample-output.png

正确率还可以,这个例子里一共是153个字符,其中4个识别错误,包括标点符号。后来经过更多的测试,我发现识别一篇文章的正确率大概在 95% 左右,主要错误在形状类似的字和标点符号上,比如大、太、夫,等等。有些字偶尔会被拆开,比如“把”会被拆成“扌”和“巴”。标点符号经常引起错误,比如“小”会和逗号连在一起错认成一个字。单个字的识别率没有测试过。考虑到我没有用任何复杂的图片处理或者是外带的数据,这样的结果还是不错的。

如果要提高正确率,一个简单的办法是成生一个常用字符表,Tesseract 在预测时可以参考这个表进行判断。另一个简单的办法是组词表,即 fixed-length-dawg。图片处理也可以提高识别率,但是如何处理需要看图像本身,在这里不好讨论。还有一个办法就是更加有针对性的训练,不过这个只是我的猜测。

除了增加行距,我发现把图片二值化(纯黑白化)也可以有效地提高准确率。如果安装了 ImageMagick,只需要一个简单命令就可以,比如:

convert input.png -threshold 50% output.png

下面给出我用的 Tesseract OCR 设置,可以放在 /usr/local/share/tessdata/config 里面,运行 tesseract 的时候用文件名调用,或包括在语言包里面。这些设置有许多是参照 Tesseract OCR 官方中文语言包里的设置。详细设置可以参考这个网页

chop_enable 0
language_model_ngram_space_delimited_language 0
chs_leading_punct            (‘“
chs_trailing_punct1           )。,;:?!
chs_trailing_punct2           )’”
textord_single_height_mode 1
textord_use_cjk_fp_model 0
segment_nonalphabetic_script 1
permute_script_word 0
textord_force_make_prop_words 1
use_new_state_cost              1
enable_new_segsearch            1
segment_segcost_rating          1
heuristic_segcost_rating_base   1.25
heuristic_weight_rating         1
assume_fixed_pitch_char_segment 1
heuristic_max_char_wh_ratio     1.3
segsearch_max_char_wh_ratio     1.9
classify_integer_matcher_multiplier 6
edges_use_new_outline_complexity    1
edges_children_fix                  1
edges_max_children_per_outline     20
edges_max_children_layers           5
edges_children_count_limit      10000
tosp_force_wordbreak_on_punct       0
textord_noise_sizelimit           0.2
textord_noise_normratio             5
textord_max_noise_size              7

最后,有兴趣的读者可以在这里体验一下:mw.gl/sites/ocr,不过速度可能有些慢。(如果要用上面例子的图片,行距是44像素。)

更改纪录

  • 2013-6-5 更新图像处理部分
  • 2016-8-2 变更体验网站地址

Setup OpenVPN on Ubuntu Server

|

This tutorial will help you setup a private OpenVPN server. By private I mean the server is intended for only one or two clients.

Before proceeding further, I assume you have some basic knowledge about computer networking and the *nix systems.

Examine the network diagram below to make sure this tutorial is suitable for you. The laptop's traffic will be routed through the server, thus it will appear as if laptop is the server. When VPN is working, your laptop's external IP should be your server's IP (google "what is my ip").

+----------+      +---------------------+               +------------------+
| INTERNET |======|{eth0  Server {tun0} |===(GATEWAY)===|   Laptop (LAN)   |
|          |      |  tun0: 10.8.0.1/24  |               |  192.168.0.0/24  |
+----------+      +---------------------+               +------------------+

Essentially it's just level 3 routing performed by OpenVPN and Ubuntu's iptables. You could also do bridging which works on level 2, but bridging is not covered in this tutorial.

To begin, install openvpn and generate the certificates and keys (server and client) by following the instructions here: OpenVPN, don't worry about any configurations yet.

Pick a memorable name for the client certificate. Suppose the client is called "fred", then KEY_CN should be "fred", and certificate file should be fred.crt and key file should be fred.key. This client name will be used for IP address assignment in OpenVPN. (See the paragraph after server config file for more.)

Start configuring openvpn from the example server.conf file. It is recommended that you take a bit of time to read through the comments and understand the settings. Sample TCP configuration file is given below.

Note that it doesn't matter whether OpenVPN is listening on TCP or UDP, all packets will be redirected. If you want to tunnel traffic over UDP, just change proto tcp to proto udp. And if you want to have the OpenVPN server listen on both UDP and TCP, you will need to create two different configuration files.

## OpenVPN server, TCP

port 1194
proto tcp
dev tun

ca ca.crt
cert server.crt
key server.key  # This file should be kept secret

dh dh1024.pem

server 10.8.0.0 255.255.255.0

# Assign IP to client by name
ifconfig-pool-persist ipp.txt
client-config-dir ccd

# Push the correct gateway to client
# Note: need iptables for NAT
push "redirect-gateway def1 bypass-dhcp"

keepalive 10 120

tls-auth ta.key 0 # This file is secret

# Select a cryptographic cipher.
# This config item must be copied to
# the client config file as well.
cipher AES-128-CBC   # AES

# Enable compression on the VPN link.
# If you enable it here, you must also
# enable it in the client config file.
comp-lzo

max-clients 3

# It's a good idea to reduce the OpenVPN
# daemon's privileges after initialization.
;user nobody
;group nogroup

# The persist options will try to avoid
# accessing certain resources on restart
# that may no longer be accessible because
# of the privilege downgrade.
persist-key
persist-tun

# Output a short status file showing
# current connections, truncated
# and rewritten every minute.
status openvpn-status.log

# Set the appropriate level of log
# file verbosity.
verb 3

There are additional files needed to assign IP address to client by name. First the IP needs to be reserved, so create ipp.txt in /etc/openvpn and paste this line in:

fred,10.8.0.4

This line will reserve 10.8.0.4 for fred. Now create directory ccd, and create file fred in ccd and paste this line in:

ifconfig-push 10.8.0.4 10.8.0.1

This line will tell the client to use 10.8.0.4 as its IP address and 10.8.0.1 as the gateway. That's all we need to make sure fred always gets assigned 10.8.0.4. The reason why you might want this is to allow WLAN connections to get to fred, which can be done easily via iptables.

Now that the OpenVPN server is configured, try to start it. If it fails, it's usually because the tun device doesn't exist. Look at the log to find out what exactly went wrong. For certain hosts you have to enable it in the control panel or ask for it to be enabled. If OpenVPN successfully starts, you should see at least one tunnel interface (tun0) when you run ifconfig.

For the client, I'm using Tunnelblick on Mac OS X 10.8.3 (as of writing) and my config file is as follows:

# Specify that we are a client and that we
# will be pulling certain config file directives
# from the server.
client
dev tun
proto tcp
remote 1.2.3.4 1194

# Keep trying indefinitely to resolve the
# host name of the OpenVPN server.  Very useful
# on machines which are not permanently connected
# to the internet such as laptops.
resolv-retry infinite

# Most clients don't need to bind to
# a specific local port number.
nobind

# Try to preserve some state across restarts.
persist-key
persist-tun

ca ca.crt
# don't forget to rename these files
cert fred.crt
key fred.key

# If a tls-auth key is used on the server
# then every client must also have the key.
tls-auth ta.key 1
cipher AES-128-CBC

# Enable compression on the VPN link.
# Don't enable this unless it is also
# enabled in the server config file.
comp-lzo

# Set log file verbosity.
verb 3

# Get the correct gateway from server
pull

If the client can connect to server, then we've almost got everything to work. At this point, you should try pinging the server from client and pinging the client from server, at least one of the pings should work. Also check that client is assigned the correct IP.

Now it is time to get the packets routed properly with iptables NAT rules.

iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o eth0 -j MASQUERADE

If your VPS uses OpenVZ you'll have venet0, then use these rules:

iptables -A FORWARD -i venet0 -o tun0 -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i tun0 -o venet0 -j ACCEPT
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -o venet0 -j SNAT --to 1.2.3.4
iptables -t nat -A POSTROUTING -j SNAT --to-source 1.2.3.4

Double check the rules are correctly added.

$ iptables -t nat -nvL

I mentioned before that you could allow external WLAN connections to get to clients within the VPN, this is done by opening a port and forward incoming traffic of that port to the client. Use this rule (assuming the client's IP is always 10.8.0.4):

iptables -t nat -A PREROUTING -p tcp -m tcp --dport 1234 -j DNAT --to-destination 10.8.0.4

If you have any confusions, this wiki page is pretty helpful: OpenVPN - ArchWiki. Also don't forget to enable IP forwarding, which is covered in that link too.

Now restart the server and have the client reconnect. Tunnelblick should automatically tell you your IP has been changed, that means your VPN is working. Double check that in the routing tables:

$ netstat -rn

Pay attention to the gateway, your server's IP should show up as the gateway, that confirms your external IP has been changed. Also, in case you have any DNS troubles, I always use Google's DNS servers: 8.8.8.8 and 8.8.4.4.

For additional security/stealth you should consider stunnel, follow the instructions here: Install and Setup OpenVPN Stealth with Stunnel on Ubuntu 12 +. It's very easy to setup, but note that it works with TCP only.

Lastly, don't forget to save the iptables rules with iptables-save. Or look at the script solution on this page: IptablesHowTo.

Last updated on 2013-08-19.

2G iPhone updating to iPhone OS 3.0

|

It's confirmed that if you have a 2G iPhone that is unlocked with BootNeuter (running firmware 2.2.1), you can safely update to OS 3.0 using iTunes (yes just press that Update button).

For detailed and step-by-step instructions see here

This method requires: jailbreaked iPhone 2.x, Cydia, Term-vt100, OpenSSH

Install MobileInstallation Patch from Cydia via “http://www.iphone.org.hk/apt/”, the patch is under section “Tweaks”. If you currently have the hackulo.us MobileInstallation, you ALSO need to install the patch. Don’t worry, you can install appulous apps with this patch.

  • In XCode, go to Project > Edit Project Settings.
  • Set Base SDK to "Device - iPhone OS 2.0"
  • Under Code Signing Identity, set Any iPhone OS Device to "Don't Code Sign"
  • Delete the value of Code Signing Resource Rules Path
  • Delete all previous build files.
  • Go to Project > Set Active SDK > Device - iPhone OS 2.0
  • Go to Project > Set Active Build Configuration > Release
  • Build. NOT Build and Go.
  • Copy MyApp.app to iPhone under /Applications using ssh or whatever method you prefer.

Now open Terminal on iPhone and execute the following code

su -
chmod +x MyApp.app
ldid -S MyApp.app/MyApp
killall SpringBoard //restart the SpringBoard

Run MyApp by touching the icon on SpringBoard.

Update: With the MobileInstallation Patch mentioned above, iPhone 2.2 seems unable to install/upgrade app by itself, you have to use iTunes (8.x) to uninstall an app first, then install/upgrade it.