2008年12月24日 星期三
2008年12月8日 星期一
The difference between AF_XXX and PF_XXX
Sometimes we are confused in using the AF_XXX and PF_XXX, like AF_INET vs PF_INET, AF_NETLINK vs PF_NETLINK. From the viewpoint of source, both them are identical.
Refer to the include/linux/socket.h. You can observe that the PF_XXX (protocol family) is defined the same as AF_XXX (address family)
/* Protocol families, same as address families. */
#define PF_UNSPEC AF_UNSPEC
#define PF_UNIX AF_UNIX
#define PF_LOCAL AF_LOCAL
#define PF_INET AF_INET
#define PF_AX25 AF_AX25
#define PF_IPX AF_IPX
#define PF_APPLETALK AF_APPLETALK
#define PF_NETROM AF_NETROM
#define PF_BRIDGE AF_BRIDGE
#define PF_ATMPVC AF_ATMPVC
#define PF_X25 AF_X25
#define PF_INET6 AF_INET6
#define PF_ROSE AF_ROSE
#define PF_DECnet AF_DECnet
#define PF_NETBEUI AF_NETBEUI
#define PF_SECURITY AF_SECURITY
#define PF_KEY AF_KEY
#define PF_NETLINK AF_NETLINK
#define PF_ROUTE AF_ROUTE
#define PF_PACKET AF_PACKET
#define PF_ASH AF_ASH
The following is the definition of the constant of each address family variable
/* Supported address families. */
#define AF_UNSPEC 0
#define AF_UNIX 1 /* Unix domain sockets */
#define AF_LOCAL 1 /* POSIX name for AF_UNIX */
#define AF_INET 2 /* Internet IP Protocol */
#define AF_AX25 3 /* Amateur Radio AX.25 */
#define AF_IPX 4 /* Novell IPX */
#define AF_APPLETALK 5 /* AppleTalk DDP */
#define AF_NETROM 6 /* Amateur Radio NET/ROM */
#define AF_BRIDGE 7 /* Multiprotocol bridge */
#define AF_ATMPVC 8 /* ATM PVCs */
#define AF_X25 9 /* Reserved for X.25 project */
#define AF_INET6 10 /* IP version 6 */
#define AF_ROSE 11 /* Amateur Radio X.25 PLP */
#define AF_DECnet 12 /* Reserved for DECnet project */
#define AF_NETBEUI 13 /* Reserved for 802.2LLC project*/
#define AF_SECURITY 14 /* Security callback pseudo AF */
#define AF_KEY 15 /* PF_KEY key management API */
#define AF_NETLINK 16
#define AF_ROUTE AF_NETLINK /* Alias to emulate 4.4BSD */
#define AF_PACKET 17 /* Packet family */
#define AF_ASH 18 /* Ash */
#define AF_ECONET 19 /* Acorn Econet */
#define AF_ATMSVC 20 /* ATM SVCs */
Refer to the include/linux/socket.h. You can observe that the PF_XXX (protocol family) is defined the same as AF_XXX (address family)
/* Protocol families, same as address families. */
#define PF_UNSPEC AF_UNSPEC
#define PF_UNIX AF_UNIX
#define PF_LOCAL AF_LOCAL
#define PF_INET AF_INET
#define PF_AX25 AF_AX25
#define PF_IPX AF_IPX
#define PF_APPLETALK AF_APPLETALK
#define PF_NETROM AF_NETROM
#define PF_BRIDGE AF_BRIDGE
#define PF_ATMPVC AF_ATMPVC
#define PF_X25 AF_X25
#define PF_INET6 AF_INET6
#define PF_ROSE AF_ROSE
#define PF_DECnet AF_DECnet
#define PF_NETBEUI AF_NETBEUI
#define PF_SECURITY AF_SECURITY
#define PF_KEY AF_KEY
#define PF_NETLINK AF_NETLINK
#define PF_ROUTE AF_ROUTE
#define PF_PACKET AF_PACKET
#define PF_ASH AF_ASH
The following is the definition of the constant of each address family variable
/* Supported address families. */
#define AF_UNSPEC 0
#define AF_UNIX 1 /* Unix domain sockets */
#define AF_LOCAL 1 /* POSIX name for AF_UNIX */
#define AF_INET 2 /* Internet IP Protocol */
#define AF_AX25 3 /* Amateur Radio AX.25 */
#define AF_IPX 4 /* Novell IPX */
#define AF_APPLETALK 5 /* AppleTalk DDP */
#define AF_NETROM 6 /* Amateur Radio NET/ROM */
#define AF_BRIDGE 7 /* Multiprotocol bridge */
#define AF_ATMPVC 8 /* ATM PVCs */
#define AF_X25 9 /* Reserved for X.25 project */
#define AF_INET6 10 /* IP version 6 */
#define AF_ROSE 11 /* Amateur Radio X.25 PLP */
#define AF_DECnet 12 /* Reserved for DECnet project */
#define AF_NETBEUI 13 /* Reserved for 802.2LLC project*/
#define AF_SECURITY 14 /* Security callback pseudo AF */
#define AF_KEY 15 /* PF_KEY key management API */
#define AF_NETLINK 16
#define AF_ROUTE AF_NETLINK /* Alias to emulate 4.4BSD */
#define AF_PACKET 17 /* Packet family */
#define AF_ASH 18 /* Ash */
#define AF_ECONET 19 /* Acorn Econet */
#define AF_ATMSVC 20 /* ATM SVCs */
Some stuff that have used netlink to pass data to kernel
Some stuff that have used netlink to pass data to kernel
* ulogd
http://www.netfilter.org/projects/ulogd/index.html
* libipq
http://www.cs.princeton.edu/~nakao/libipq.htm
* ulogd
http://www.netfilter.org/projects/ulogd/index.html
* libipq
http://www.cs.princeton.edu/~nakao/libipq.htm
2008年12月7日 星期日
Some Makefile tips and questions
$(SHARED_LIBS:%.so=%.d): %.d: %.c
@-$(CC) -M -MG $(CFLAGS) $< | \
sed -e 's@^.*\.o:@$*.d $*_sh.o:@' > $@
Q. What doest this mean? $(SHARED_LIBS:%.so=%.d)
Q. What does here two colon mean?
-------------------------------------------------------
Makefile Rule Format:
In general, a rule looks like this:
targets : prerequisites
command
...
or like this:
targets : prerequisites ; command
command
-------------------------------------------------------
Another similar parts.
$(SHARED_LIBS): %.so : %_sh.o
$(CC) -shared $(EXT_LDFLAGS) -o $@ $<
$(SHARED_SE_LIBS:%.so=%.d): %.d: %.c
@-$(CC) -M -MG $(CFLAGS) $< | \
sed -e 's@^.*\.o:@$*.d $*_sh.o:@' > $@
$(SHARED_SE_LIBS): %.so : %_sh.o
$(LD) -shared $(EXT_LDFLAGS) -o $@ $< $(LDLIBS)
@-$(CC) -M -MG $(CFLAGS) $< | \
sed -e 's@^.*\.o:@$*.d $*_sh.o:@' > $@
Q. What doest this mean? $(SHARED_LIBS:%.so=%.d)
Q. What does here two colon mean?
-------------------------------------------------------
Makefile Rule Format:
In general, a rule looks like this:
targets : prerequisites
command
...
or like this:
targets : prerequisites ; command
command
-------------------------------------------------------
Another similar parts.
$(SHARED_LIBS): %.so : %_sh.o
$(CC) -shared $(EXT_LDFLAGS) -o $@ $<
$(SHARED_SE_LIBS:%.so=%.d): %.d: %.c
@-$(CC) -M -MG $(CFLAGS) $< | \
sed -e 's@^.*\.o:@$*.d $*_sh.o:@' > $@
$(SHARED_SE_LIBS): %.so : %_sh.o
$(LD) -shared $(EXT_LDFLAGS) -o $@ $< $(LDLIBS)
2008年12月3日 星期三
vmware config process
Run /usr/bin/vmware-config.pl to config the vMware interface settings
Would you like to skip networking setup and keep your old settings as they are?
(yes/no) [yes] no
Do you want networking for your virtual machines? (yes/no/help) [yes]
Would you prefer to modify your existing networking configuration using the
wizard or the editor? (wizard/editor/help) [wizard] editor ==> use editor to add the vmware vmnet <==> physical interface mapping
The following virtual networks have been defined:
. vmnet0 is bridged to eth0
. vmnet1 is a host-only network on private subnet 172.16.198.0.
. vmnet2 is bridged to eth1
. vmnet3 is bridged to eth2
. vmnet8 is a NAT network on private subnet 172.16.126.0.
Do you wish to make any changes to the current virtual networks settings?
(yes/no) [no] yes
Which virtual network do you wish to configure? (0-99) 4
What type of virtual network do you wish to set vmnet4?
(bridged,hostonly,nat,none) [none] bridged
Configuring a bridged network for vmnet4.
Your computer has multiple ethernet network interfaces available: eth3, eth4,
eth5, eth6, eth7, eth8. Which one do you want to bridge to vmnet4? [eth0] eth3
The following virtual networks have been defined:
. vmnet0 is bridged to eth0
. vmnet1 is a host-only network on private subnet 172.16.198.0.
. vmnet2 is bridged to eth1
. vmnet3 is bridged to eth2
. vmnet4 is bridged to eth3 ==> The new added one interface
. vmnet8 is a NAT network on private subnet 172.16.126.0.
Would you like to skip networking setup and keep your old settings as they are?
(yes/no) [yes] no
Do you want networking for your virtual machines? (yes/no/help) [yes]
Would you prefer to modify your existing networking configuration using the
wizard or the editor? (wizard/editor/help) [wizard] editor ==> use editor to add the vmware vmnet <==> physical interface mapping
The following virtual networks have been defined:
. vmnet0 is bridged to eth0
. vmnet1 is a host-only network on private subnet 172.16.198.0.
. vmnet2 is bridged to eth1
. vmnet3 is bridged to eth2
. vmnet8 is a NAT network on private subnet 172.16.126.0.
Do you wish to make any changes to the current virtual networks settings?
(yes/no) [no] yes
Which virtual network do you wish to configure? (0-99) 4
What type of virtual network do you wish to set vmnet4?
(bridged,hostonly,nat,none) [none] bridged
Configuring a bridged network for vmnet4.
Your computer has multiple ethernet network interfaces available: eth3, eth4,
eth5, eth6, eth7, eth8. Which one do you want to bridge to vmnet4? [eth0] eth3
The following virtual networks have been defined:
. vmnet0 is bridged to eth0
. vmnet1 is a host-only network on private subnet 172.16.198.0.
. vmnet2 is bridged to eth1
. vmnet3 is bridged to eth2
. vmnet4 is bridged to eth3 ==> The new added one interface
. vmnet8 is a NAT network on private subnet 172.16.126.0.
vmware daemon
>> If we show the vmware process, we can know there are some vmware daemons provide the services
- vmnet-natd
always exist, even no vmware client images
- vmware-serverd
always exist, even no vmware client images
- vmnet-dhcpd
provide the dhcp service, only service in the vmnet1 (Host-only) and vmnet8 (NAT)
- vmware-vmx
For each vmware client image, the vmware will fork a vmware-vmx to provide the service
# ps auxwww | grep vmware
root 3101 0.0 0.0 1756 356 ? Ss Nov07 0:00 /usr/bin/vmnet-natd -d /var/run/vmnet-natd-8.pid -m /var/run/vmnet-natd-8.mac -c /etc/vmware/vmnet8/nat/nat.conf
root 3107 0.0 0.3 17336 14800 ? Ss Nov07 2:45 /usr/sbin/vmware-serverd -s -d
root 3629 0.0 0.0 1756 232 ? Ss Nov07 0:00 /usr/bin/vmnet-dhcpd -cf /etc/vmware/vmnet1/dhcpd/dhcpd.conf -lf /etc/vmware/vmnet1/dhcpd/dhcpd.leases -pf /var/run/vmnet-dhcpd-vmnet1.pid vmnet1
root 3630 0.0 0.0 1756 232 ? Ss Nov07 0:00 /usr/bin/vmnet-dhcpd -cf /etc/vmware/vmnet8/dhcpd/dhcpd.conf -lf /etc/vmware/vmnet8/dhcpd/dhcpd.leases -pf /var/run/vmnet-dhcpd-vmnet8.pid vmnet8
root 3710 10.6 7.2 404380 292520 ? S
- vmnet-natd
always exist, even no vmware client images
- vmware-serverd
always exist, even no vmware client images
- vmnet-dhcpd
provide the dhcp service, only service in the vmnet1 (Host-only) and vmnet8 (NAT)
- vmware-vmx
For each vmware client image, the vmware will fork a vmware-vmx to provide the service
# ps auxwww | grep vmware
root 3101 0.0 0.0 1756 356 ? Ss Nov07 0:00 /usr/bin/vmnet-natd -d /var/run/vmnet-natd-8.pid -m /var/run/vmnet-natd-8.mac -c /etc/vmware/vmnet8/nat/nat.conf
root 3107 0.0 0.3 17336 14800 ? Ss Nov07 2:45 /usr/sbin/vmware-serverd -s -d
root 3629 0.0 0.0 1756 232 ? Ss Nov07 0:00 /usr/bin/vmnet-dhcpd -cf /etc/vmware/vmnet1/dhcpd/dhcpd.conf -lf /etc/vmware/vmnet1/dhcpd/dhcpd.leases -pf /var/run/vmnet-dhcpd-vmnet1.pid vmnet1
root 3630 0.0 0.0 1756 232 ? Ss Nov07 0:00 /usr/bin/vmnet-dhcpd -cf /etc/vmware/vmnet8/dhcpd/dhcpd.conf -lf /etc/vmware/vmnet8/dhcpd/dhcpd.leases -pf /var/run/vmnet-dhcpd-vmnet8.pid vmnet8
root 3710 10.6 7.2 404380 292520 ? S
2008年12月1日 星期一
hexdump experience
>> Show the hex and ascill data of a specified file
# hexdump -C fingerprint.dat
00000000 0a 00 00 00 49 2c e7 24 61 6c 65 78 31 30 30 00 |....I,.$alex100.|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 01 00 00 00 8c 71 66 aa |.............qf.|
00000030 00 00 00 00 01 00 00 00 |........|
00000038
# hexdump -C fingerprint.dat
00000000 0a 00 00 00 49 2c e7 24 61 6c 65 78 31 30 30 00 |....I,.$alex100.|
00000010 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |................|
00000020 00 00 00 00 00 00 00 00 01 00 00 00 8c 71 66 aa |.............qf.|
00000030 00 00 00 00 01 00 00 00 |........|
00000038
2008年11月30日 星期日
static/dynamic library, "file" comman usage
Refer from https://www.cs.utk.edu/help/doku.php?id=compile:c
Compile Phase:
# gcc -c hello.c
Dynamic Linking:
# gcc -o hello hello.o
# ls -lh hello
-rwx------ 1 user group 12K Sep 18 18:35 hello
# file hello
hello: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
for GNU/Linux 2.2.0, dynamically linked (uses shared libs), not stripped
Static Linking:
# gcc -static -o hello hello.o
# ls -lh hello
# -rwx------ 1 user group 465K Sep 18 18:38 hello
# file hello
hello: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV),
for GNU/Linux 2.2.0, statically linked, not stripped
2008年11月27日 星期四
How to interpret the tcpdump header
15:19:03.018317 IP (tos 0x0, ttl 63, id 27496, offset 0, flags [DF], proto TCP (6), length 62) 140.113.102.165.44298 > 192.168.2.107.46877: P, cksum 0x0180 (correct), 1:11(10) ack 12 win 365
0x0000: 4500 003e 6b68 4000 3f06 1a28 8c71 66a5 E..>kh@.?..(.qf.
0x0010: c0a8 026b ad0a b71d 6183 1a2f 7d66 3155 ...k....a../}f1U
0x0020: 8018 016d 0180 0000 0101 080a 8c57 7f5a ...m.........W.Z
0x0030: a044 62e7 6865 6c6c 6f77 6f72 6c64 .Db.helloworld
15:19:03.018317 ==> timestamp
IP (tos 0x0, ttl 63, id 27496, offset 0, flags [DF], proto TCP (6), length 62)
==> ip header
tos 0x0 ==> tos value 0x0
ttl 63 ==> ttl value is 63
id 27496==> ip id is 27496
offset 0 ==> offset is 0
flags [DF] ==> IP flag don't flagment
proto TCP (6) ==> trasnport protocol is tcp (6)
length 62) ==> ip length is 62 (include ip header)
140.113.102.165.44298 > 192.168.2.107.46877
==> source ip/port from 140.113.102.165.44298 to destination ip/port 192.168.2.107.46877
: P, cksum 0x0180 (correct), 1:11(10) ack 12 win 365
:P ==> TCP flag, PUSH flag is on, ACK will not show, S indicate SYN, F indicate FIN, R indicate RST
cksum 0x0180 (correct) ==> tcp check sum is 0x0180, validate it is correct
1:11(10) ==> sequence number diff is from 1:11 total 10 bytes
ack 12 ==> the data had been received is 12-1 bytes, next expected bytes is 12th byte
win 365 ==>
0101 080a 8c57 7f5a a044 62e7
01 => no operation, nop
01 => no operation, nop
08 => timestamp
0a => 10 bytes (in the tomestamp option, 8 bytes for data)
8c57 7f5a => Timestamp value, 2354544474
a044 62e7 => Timestamp echo reply, 2688836327
0x0000: 4500 003e 6b68 4000 3f06 1a28 8c71 66a5 E..>kh@.?..(.qf.
0x0010: c0a8 026b ad0a b71d 6183 1a2f 7d66 3155 ...k....a../}f1U
0x0020: 8018 016d 0180 0000 0101 080a 8c57 7f5a ...m.........W.Z
0x0030: a044 62e7 6865 6c6c 6f77 6f72 6c64 .Db.helloworld
15:19:03.018317 ==> timestamp
IP (tos 0x0, ttl 63, id 27496, offset 0, flags [DF], proto TCP (6), length 62)
==> ip header
tos 0x0 ==> tos value 0x0
ttl 63 ==> ttl value is 63
id 27496==> ip id is 27496
offset 0 ==> offset is 0
flags [DF] ==> IP flag don't flagment
proto TCP (6) ==> trasnport protocol is tcp (6)
length 62) ==> ip length is 62 (include ip header)
140.113.102.165.44298 > 192.168.2.107.46877
==> source ip/port from 140.113.102.165.44298 to destination ip/port 192.168.2.107.46877
: P, cksum 0x0180 (correct), 1:11(10) ack 12 win 365
:P ==> TCP flag, PUSH flag is on, ACK will not show, S indicate SYN, F indicate FIN, R indicate RST
cksum 0x0180 (correct) ==> tcp check sum is 0x0180, validate it is correct
1:11(10) ==> sequence number diff is from 1:11 total 10 bytes
ack 12 ==> the data had been received is 12-1 bytes, next expected bytes is 12th byte
win 365 ==>
0101 080a 8c57 7f5a a044 62e7
01 => no operation, nop
01 => no operation, nop
08 => timestamp
0a => 10 bytes (in the tomestamp option, 8 bytes for data)
8c57 7f5a => Timestamp value, 2354544474
a044 62e7 => Timestamp echo reply, 2688836327
2008年11月20日 星期四
Linux Command - fdisk, File system management
>> partition the hard drive, list all the partition tables
# fdisk -l
>> Enter specified partions, press p to display all the partions
# fdisk /dev/sda
Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000dda97
Device Boot Start End Blocks Id System
/dev/sda1 * 1 25 200781 83 Linux
/dev/sda2 26 1044 8185117+ 8e Linux LVM ==> LVM is a Linux Volumn Manager, Can support RAID architecture
>> format the file system
# mke2fs
>> physically mount the file system
# mount /dev/hdc1 /vm
# fdisk -l
>> Enter specified partions, press p to display all the partions
# fdisk /dev/sda
Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x000dda97
Device Boot Start End Blocks Id System
/dev/sda1 * 1 25 200781 83 Linux
/dev/sda2 26 1044 8185117+ 8e Linux LVM ==> LVM is a Linux Volumn Manager, Can support RAID architecture
>> format the file system
# mke2fs
>> physically mount the file system
# mount /dev/hdc1 /vm
Linux System - File Systems.. LVM management
# ls -la /dev/rootvg/
/dev/rootvg/LogVol00 -> /dev/mapper/rootvg-LogVol00
/dev/rootvg/LogVol01 -> /dev/mapper/rootvg-LogVol01
/dev/rootvg/LogVolVM01 -> /dev/mapper/rootvg-LogVolVM01
# ls -la /dev/mapper/
total 0
control
rootvg-LogVol00
rootvg-LogVol01
rootvg-LogVolVM01
# lvdisplay /dev/rootvg/LogVol00
--- Logical volume ---
LV Name /dev/rootvg/LogVol00
VG Name rootvg
LV UUID SYotdp-V3oQ-Gc5p-wDjv-txyz-Q09p-7cgsXz
LV Write Access read/write
LV Status available
# open 1
LV Size 200.00 GB
Current LE 6400
Segments 2
Allocation inherit
Read ahead sectors 0
Block device 253:0
# lvdisplay /dev/rootvg/LogVol01
--- Logical volume ---
LV Name /dev/rootvg/LogVol01
VG Name rootvg
LV UUID 8MT67X-WpcY-Nyw3-MA57-jKKn-ryq6-9dMWcD
LV Write Access read/write
LV Status available
# open 1
LV Size 1.94 GB
Current LE 62
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:1
# lvdisplay /dev/rootvg/LogVolVM01
--- Logical volume ---
LV Name /dev/rootvg/LogVolVM01
VG Name rootvg
LV UUID e9LYTF-yFRz-RdC1-KC8q-wAN1-2UiM-8dILP5
LV Write Access read/write
LV Status available
# open 0
LV Size 50.00 GB
Current LE 1600
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:2
How to enlarge an exsiting partition, here is my experience.
1. First umount the file system
# umount /vm
2. Then use lvextend to extend the logical volumn to the specified volumn size
# lvextend -L80G /dev/rootvg/LogVolVM01
3. Then run the e2fsck to check the new file system
# e2fsck -f /dev/rootvg/LogVolVM01
4. Use resize2fs to reorgnize the new logical volumn
# resize2fs /dev/rootvg/LogVolVM01
5. mount the logical volumn to the file sytem
# mount /dev/rootvg/LogVolVM01 /vm
http://www.kume.idv.tw/read-226.html
首先，我要說明一下。我用的是Fedora Code 5。所以我以下所用到的指令，全部都有。不需要download 與build任何東西。
第一件事就是載入LVM所需要的module
% modprobe dm-mod ==> 我的不work, 可能是沒有module
接著就是找出partiton中的VG(Volume Group)，以我的狀況為例，我將硬碟以usb連接到電腦後，由dmesg可以知道有 /dev/sda1, /dev/sda2兩個patitons，在以 fdisk -l /dev/sda 可以知道/dev/sda2是LVM格式。接著，我就執行：
% pvscan ==> Fedora Rescue disc 沒有包含這個程式
PV /dev/sda2 VG VolGroup00 lvm2 [18.50 GB / 32.00 MB free]
Total: 1 [18.50 GB] / in use: 1 [18.50 GB] / in no VG: 0 [0 ]
從以上結果，我可以知道我有一個VG，叫做VolGroup00，他的PV(Physical Volume)是 /dev/sda2
接著我就執行：
% vgchange -ay VolGroup00 ==>我測試是不用加VolGroup00 參數, 只要% cgchange -ay 就會在/dev/ 下面產生一個VolGroup00 folder, 並且在這個目錄下產生該有的device（其實是link到/dev /mapper下）。
% ls /dev/VolGroup00/
LogVol00 LogVol01
這時候vgchange會在/dev下產生一個VolGroup00的目錄，並且在這個目錄下產生該有的device（其實是link到/dev /mapper下）。然後我利用 ls就可以知道VolGroup00這個VG下有兩個LV(Logical Volume)。分別是LogVol00 與 LogVol01。然後我只要mount我要的就可以了。
% mkdir /mnt/usb
% mount /dev/VolGroup00/LogVol00 /mnt/usb/ ==> 在/dev/VolGroup00/ folder下面會有兩個file, 分別表示LVM的2個檔案, 包含有 /dev/VolGroup00/LogVol00, /dev/VolGroup00/LogVol01, 這裡的例子是將第一個LVM mount 到/mnt/usb 這個logic folder 下, 然後就可以直接存取 /mnt/usb 這個路徑去抓LVM 1的data了
Q. How to rescue the HD with LVM file system?
A.
1. Insert the Fedora disc 1, enter rescue mode.
2. Issue the following commands to find the LVM partitions
# lvm vgscan
# lvm vgchange -ay
# lvm lvs
// e2fsck -y==>all questions response "yes", -c==>fix bad block file system
# e2fsck -y -c /dev/VolGroup00/LogVol00
# e2fsck -y -c /dev/VolGroup00/LogVol01
/dev/rootvg/LogVol00 -> /dev/mapper/rootvg-LogVol00
/dev/rootvg/LogVol01 -> /dev/mapper/rootvg-LogVol01
/dev/rootvg/LogVolVM01 -> /dev/mapper/rootvg-LogVolVM01
# ls -la /dev/mapper/
total 0
control
rootvg-LogVol00
rootvg-LogVol01
rootvg-LogVolVM01
# lvdisplay /dev/rootvg/LogVol00
--- Logical volume ---
LV Name /dev/rootvg/LogVol00
VG Name rootvg
LV UUID SYotdp-V3oQ-Gc5p-wDjv-txyz-Q09p-7cgsXz
LV Write Access read/write
LV Status available
# open 1
LV Size 200.00 GB
Current LE 6400
Segments 2
Allocation inherit
Read ahead sectors 0
Block device 253:0
# lvdisplay /dev/rootvg/LogVol01
--- Logical volume ---
LV Name /dev/rootvg/LogVol01
VG Name rootvg
LV UUID 8MT67X-WpcY-Nyw3-MA57-jKKn-ryq6-9dMWcD
LV Write Access read/write
LV Status available
# open 1
LV Size 1.94 GB
Current LE 62
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:1
# lvdisplay /dev/rootvg/LogVolVM01
--- Logical volume ---
LV Name /dev/rootvg/LogVolVM01
VG Name rootvg
LV UUID e9LYTF-yFRz-RdC1-KC8q-wAN1-2UiM-8dILP5
LV Write Access read/write
LV Status available
# open 0
LV Size 50.00 GB
Current LE 1600
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:2
How to enlarge an exsiting partition, here is my experience.
1. First umount the file system
# umount /vm
2. Then use lvextend to extend the logical volumn to the specified volumn size
# lvextend -L80G /dev/rootvg/LogVolVM01
3. Then run the e2fsck to check the new file system
# e2fsck -f /dev/rootvg/LogVolVM01
4. Use resize2fs to reorgnize the new logical volumn
# resize2fs /dev/rootvg/LogVolVM01
5. mount the logical volumn to the file sytem
# mount /dev/rootvg/LogVolVM01 /vm
http://www.kume.idv.tw/read-226.html
首先，我要說明一下。我用的是Fedora Code 5。所以我以下所用到的指令，全部都有。不需要download 與build任何東西。
第一件事就是載入LVM所需要的module
% modprobe dm-mod ==> 我的不work, 可能是沒有module
接著就是找出partiton中的VG(Volume Group)，以我的狀況為例，我將硬碟以usb連接到電腦後，由dmesg可以知道有 /dev/sda1, /dev/sda2兩個patitons，在以 fdisk -l /dev/sda 可以知道/dev/sda2是LVM格式。接著，我就執行：
% pvscan ==> Fedora Rescue disc 沒有包含這個程式
PV /dev/sda2 VG VolGroup00 lvm2 [18.50 GB / 32.00 MB free]
Total: 1 [18.50 GB] / in use: 1 [18.50 GB] / in no VG: 0 [0 ]
從以上結果，我可以知道我有一個VG，叫做VolGroup00，他的PV(Physical Volume)是 /dev/sda2
接著我就執行：
% vgchange -ay VolGroup00 ==>我測試是不用加VolGroup00 參數, 只要% cgchange -ay 就會在/dev/ 下面產生一個VolGroup00 folder, 並且在這個目錄下產生該有的device（其實是link到/dev /mapper下）。
% ls /dev/VolGroup00/
LogVol00 LogVol01
這時候vgchange會在/dev下產生一個VolGroup00的目錄，並且在這個目錄下產生該有的device（其實是link到/dev /mapper下）。然後我利用 ls就可以知道VolGroup00這個VG下有兩個LV(Logical Volume)。分別是LogVol00 與 LogVol01。然後我只要mount我要的就可以了。
% mkdir /mnt/usb
% mount /dev/VolGroup00/LogVol00 /mnt/usb/ ==> 在/dev/VolGroup00/ folder下面會有兩個file, 分別表示LVM的2個檔案, 包含有 /dev/VolGroup00/LogVol00, /dev/VolGroup00/LogVol01, 這裡的例子是將第一個LVM mount 到/mnt/usb 這個logic folder 下, 然後就可以直接存取 /mnt/usb 這個路徑去抓LVM 1的data了
Q. How to rescue the HD with LVM file system?
A.
1. Insert the Fedora disc 1, enter rescue mode.
2. Issue the following commands to find the LVM partitions
# lvm vgscan
# lvm vgchange -ay
# lvm lvs
// e2fsck -y==>all questions response "yes", -c==>fix bad block file system
# e2fsck -y -c /dev/VolGroup00/LogVol00
# e2fsck -y -c /dev/VolGroup00/LogVol01
2008年11月18日 星期二
kamailio doesn't forward the ACK
UAC
1. Receive 200 OK
SIP/2.0 200 OK
| INFO3 | cb_rcv2xx (id=2)^M
| INFO1 | cb_nict_kill_transaction (id=2)^M
| INFO4 | sipevent evt: method called!
2. Send ACK
3. Receive 200 OK again
| INFO1 | 2xx restransmission receveid.^M
| INFO1 | Message sent:
4. Send ACK again
| INFO1 | ACK restransmission sent.
1. Receive 200 OK
SIP/2.0 200 OK
| INFO3 |
| INFO1 |
| INFO4 |
2. Send ACK
3. Receive 200 OK again
| INFO1 |
| INFO1 |
4. Send ACK again
| INFO1 |
2008年11月17日 星期一
SIP Register with authentication required (using eXosip2)
->Send Register
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
->Send Register with header
Proxy-Authorization: Digest username="XXXXX", realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX", uri="sip:W.X.Y.Z", response="ZZZZe5d430141e984ff50fa560bWWWW", algorithm=MD5 ==>1st try
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
->Send Register with header
Proxy-Authorization: Digest username="XXXXX", realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX", uri="sip:W.X.Y.Z", response="ZZZZe5d430141e984ff50fa560bWWWW", algorithm=MD5 ==>2nd try
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
->Send Register with header
Proxy-Authorization: Digest username="XXXXX", realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX", uri="sip:W.X.Y.Z", response="ZZZZe5d430141e984ff50fa560bWWWW", algorithm=MD5 ==>3rd try
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
->Send Register with header
Proxy-Authorization: Digest username="XXXXX", realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX", uri="sip:W.X.Y.Z", response="ZZZZe5d430141e984ff50fa560bWWWW", algorithm=MD5 ==>1st try
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
->Send Register with header
Proxy-Authorization: Digest username="XXXXX", realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX", uri="sip:W.X.Y.Z", response="ZZZZe5d430141e984ff50fa560bWWWW", algorithm=MD5 ==>2nd try
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
->Send Register with header
Proxy-Authorization: Digest username="XXXXX", realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX", uri="sip:W.X.Y.Z", response="ZZZZe5d430141e984ff50fa560bWWWW", algorithm=MD5 ==>3rd try
->Receive SIP/2.0 407 Proxy Authentication Required with header
Proxy-Authenticate: Digest realm="W.X.Y.Z", nonce="YYYY0992000000046dd64e657218e5a77472ae059c69XXXX"
2008年11月16日 星期日
echo/warning/error messages in the bash shell
Reference from the kamailio source, scripts/kamdbctl.base
mdbg() {
if [ "0$VERBOSE" -ne 0 ] ; then
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e "\033[1m$1\033[0m"
else
echo "$1"
fi
fi
}
mwarn() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e '\E[37;32m'"\033[1mWARNING: $1\033[0m"
else
echo "** WARNING: $1"
fi
}
minfo() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e '\E[37;33m'"\033[1mINFO: $1\033[0m"
else
echo "** INFO: $1"
fi
}
mecho() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e "\033[1m$1\033[0m"
else
echo "$1"
fi
}
merr() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e '\E[37;31m'"\033[1mERROR: $1\033[0m"
else
echo "** ERROR: $1"
fi
}
mdbg() {
if [ "0$VERBOSE" -ne 0 ] ; then
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e "\033[1m$1\033[0m"
else
echo "$1"
fi
fi
}
mwarn() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e '\E[37;32m'"\033[1mWARNING: $1\033[0m"
else
echo "** WARNING: $1"
fi
}
minfo() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e '\E[37;33m'"\033[1mINFO: $1\033[0m"
else
echo "** INFO: $1"
fi
}
mecho() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e "\033[1m$1\033[0m"
else
echo "$1"
fi
}
merr() {
if [ -t 1 -a -z "$NOHLPRINT" ] ; then
echo -e '\E[37;31m'"\033[1mERROR: $1\033[0m"
else
echo "** ERROR: $1"
fi
}
Show the mysql information
# /usr/bin/mysqladmin version
Server version 5.0.45
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/lib/mysql/mysql.sock
Uptime: 13 sec
Threads: 1 Questions: 1 Slow queries: 0 Opens: 12 Flush tables: 1 Open tables: 6 Queries per second avg: 0.077
PS. Sometimes mysqladmin find the wrong socket (althought that socket is specified in the /etc/my.conf)
In this case we must specify the mysql socket
# mysqladmin -S /var/lib/mysql/mysql.sock version
Or we can modify the config file /etc/my.cnf
/etc/my.cnf is list below
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[client] ==> Add this two lines to make all the mysql* client command can read correct path of mysql domain socket file
socket=/var/lib/mysql/mysql.sock
>> Show all the mysql database
# /usr/bin/mysqlshow
+-------------------- +
| Databases |
+-------------------- +
| information_schema |
| mysql |
| test |
+--------------------+
>> Show all the tables of a specified mysql database (mysql)
# mysqlshow mysql
Server version 5.0.45
Protocol version 10
Connection Localhost via UNIX socket
UNIX socket /var/lib/mysql/mysql.sock
Uptime: 13 sec
Threads: 1 Questions: 1 Slow queries: 0 Opens: 12 Flush tables: 1 Open tables: 6 Queries per second avg: 0.077
PS. Sometimes mysqladmin find the wrong socket (althought that socket is specified in the /etc/my.conf)
In this case we must specify the mysql socket
# mysqladmin -S /var/lib/mysql/mysql.sock version
Or we can modify the config file /etc/my.cnf
/etc/my.cnf is list below
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Default to using old password format for compatibility with mysql 3.x
# clients (those using the mysqlclient10 compatibility package).
old_passwords=1
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[client] ==> Add this two lines to make all the mysql* client command can read correct path of mysql domain socket file
socket=/var/lib/mysql/mysql.sock
>> Show all the mysql database
# /usr/bin/mysqlshow
+-------------------- +
| Databases |
+-------------------- +
| information_schema |
| mysql |
| test |
+--------------------+
>> Show all the tables of a specified mysql database (mysql)
# mysqlshow mysql
2008年11月12日 星期三
ICMP Redirect real experience
Sometimes the source host send a packet to a router(usually default gateway) that is originally can route by the source host itself. And doesn't need the help of the router, then the router will probably send the "icmp redirect" packet to inform the source host (if the router support and enable the feature).
The following is my experience
Network topology
Source host(A/Ma)---------
|
|------Default Gateway (G/Mg)
|
Destination host(B/Mb)----
The source host (A), MAC address is Ma, and destination host (B), MAC address is Mb, is in the same subnet which all connect to the default gateway (G), MAC address is Mg.
If the A want to send a packet to the B, the packet in theory must be A/Ma-->B/Mb,
but cause of some reason, the packet become A/Ma-->B/Mg (the routing table of A is wrong, the A fill the wrong MAC address ....).
So the packet was sent to the default gateway G because of the destination MAC is Mg. And G will according the destination address of packet to lookup the routing table of the G and determine that this packet should not send to the G. Because of the source host A can send the packet directly and no need to send the packet to the G for the routing help. So the G will generate a icmp redirect packet to inform the source host this situation.
The icmp redirect packet is like following
Type: 5 (Redirect)
Code: 1 (Redirect for host)
Gateway address: B (B)
The remaining is the original packet data include ip, transport udp/tcp, app
And then the source host A will resend the packet with A/Mg-->B/Mb. And the packet will correctly send to the destination host B for processing.
In the further packets, the source host A still send the packet to the default gateway (with destination MAC address Mg). And this packet will drop by the M. But the source host will "automatically" send the correct packet to the destination B with MAC address Mb without the default gateway G sending the icmp redirect packet to inform source host A.
Every period time, the default gateway will send the icmp redirect packet to inform the source host A the wrong destination MAC addreess if the default gateway still receive the MAC-confused packet. I think that Default gateway doesn't send the icmp redirect packet each time most because of that this may be a DoS attack. So the Gateway must sensitive to response the icmp redirect packet to avoid system too busy to operate the normal tasks.
The following is my experience
Network topology
Source host(A/Ma)---------
|
|------Default Gateway (G/Mg)
|
Destination host(B/Mb)----
The source host (A), MAC address is Ma, and destination host (B), MAC address is Mb, is in the same subnet which all connect to the default gateway (G), MAC address is Mg.
If the A want to send a packet to the B, the packet in theory must be A/Ma-->B/Mb,
but cause of some reason, the packet become A/Ma-->B/Mg (the routing table of A is wrong, the A fill the wrong MAC address ....).
So the packet was sent to the default gateway G because of the destination MAC is Mg. And G will according the destination address of packet to lookup the routing table of the G and determine that this packet should not send to the G. Because of the source host A can send the packet directly and no need to send the packet to the G for the routing help. So the G will generate a icmp redirect packet to inform the source host this situation.
The icmp redirect packet is like following
Type: 5 (Redirect)
Code: 1 (Redirect for host)
Gateway address: B (B)
The remaining is the original packet data include ip, transport udp/tcp, app
And then the source host A will resend the packet with A/Mg-->B/Mb. And the packet will correctly send to the destination host B for processing.
In the further packets, the source host A still send the packet to the default gateway (with destination MAC address Mg). And this packet will drop by the M. But the source host will "automatically" send the correct packet to the destination B with MAC address Mb without the default gateway G sending the icmp redirect packet to inform source host A.
Every period time, the default gateway will send the icmp redirect packet to inform the source host A the wrong destination MAC addreess if the default gateway still receive the MAC-confused packet. I think that Default gateway doesn't send the icmp redirect packet each time most because of that this may be a DoS attack. So the Gateway must sensitive to response the icmp redirect packet to avoid system too busy to operate the normal tasks.
2008年11月9日 星期日
kamctl usage
usr/local/sbin/kamctl
Existing commands:
-- command 'start|stop|restart'
restart ............................ restart Kamailio
start .............................. start Kamailio
stop ............................... stop Kamailio
-- command 'acl' - manage access control lists (acl)
acl show [] .............. show user membership
acl grant ....... grant user membership (*)
acl revoke [] .... grant user membership(s) (*)
-- command 'lcr' - manage least cost routes (lcr)
* IP addresses must be entered in dotted quad format e.g. 1.2.3.4 *
* and must be entered in integer or text,*
* e.g. transport '2' is identical to transport 'tcp'. *
* scheme: 1=sip, 2=sips; transport: 1=udp, 2=tcp, 3=tls *
* Examples: lcr addgw level3 1.2.3.4 5080 sip tcp 1 *
* lcr addroute +1 '' 1 1 *
lcr show .....................................................................
............. show routes, gateways and groups
lcr reload ...................................................................
............. reload lcr gateways
lcr addgw
............... add a gateway with flags, tag and strip ............
................(flags, tag, and strip are optional arguments) .....
lcr rmgw ..........................................................
............... delete a gateway
lcr addroute .................................
.............. add a route ( use '' to match anything in )
lcr rmroute .................................
.............. delete a route
-- command 'cr' - manage carrierroute tables
cr show ....................................................... show tables
cr reload ..................................................... reload tables
cr dump ....................................................... show in memory tables
cr addrt ..................... add a tree
cr rmrt ....................................... rm a tree
cr addcarrier ................
...............
.........................add a carrier
(prob, strip, rewrite_prefix, rewrite_suffix,...................
flags, mask and comment are optional arguments) ...............
cr rmcarrier ................ rm a carrier
-- command 'rpid' - manage Remote-Party-ID (RPID)
rpid add ......... add rpid for a user (*)
rpid rm ................. set rpid to NULL for a user (*)
rpid show ............... show rpid of a user
-- command 'add|passwd|rm' - manage subscribers
add .......... add a new subscriber (*)
passwd ......... change user's password (*)
rm ...................... delete a user (*)
-- command 'add|dump|reload|rm|show' - manage trusted
trusted show ...................... show db content
trusted dump ...................... show cache content
trusted reload .................... reload db table into cache
trusted add
....................... add a new entry
....................... (from_pattern and tag are optional arguments)
trusted rm ............... remove all entres for the given src_ip
-- command 'dispatcher' - manage dispatcher
* Examples: dispatcher addgw 1 sip:1.2.3.1:5050 1 'outbound gateway'
* dispatcher addgw 2 sip:1.2.3.4:5050 3 ''
* dispatcher rmgw 4
dispatcher show ..................... show dispatcher gateways
dispatcher reload ................... reload dispatcher gateways
dispatcher dump ..................... show in memory dispatcher gateways
dispatcher addgw
.......................... add gateway
dispatcher rmgw ................ delete gateway
-- command 'db' - database operations
db exec ..................... execute SQL query
db roexec ................. execute read-only SQL query
db run ......................... execute SQL query from $id variable
db rorun ....................... execute read-only SQL query from
$id variable
db show ..................... display table content
Existing commands:
-- command 'start|stop|restart'
restart ............................ restart Kamailio
start .............................. start Kamailio
stop ............................... stop Kamailio
-- command 'acl' - manage access control lists (acl)
acl show [
acl grant
acl revoke
-- command 'lcr' - manage least cost routes (lcr)
* IP addresses must be entered in dotted quad format e.g. 1.2.3.4 *
*
* e.g. transport '2' is identical to transport 'tcp'. *
* scheme: 1=sip, 2=sips; transport: 1=udp, 2=tcp, 3=tls *
* Examples: lcr addgw level3 1.2.3.4 5080 sip tcp 1 *
* lcr addroute +1 '' 1 1 *
lcr show .....................................................................
............. show routes, gateways and groups
lcr reload ...................................................................
............. reload lcr gateways
lcr addgw
............... add a gateway with flags, tag and strip ............
................(flags, tag, and strip are optional arguments) .....
lcr rmgw
............... delete a gateway
lcr addroute
.............. add a route ( use '' to match anything in
lcr rmroute
.............. delete a route
-- command 'cr' - manage carrierroute tables
cr show ....................................................... show tables
cr reload ..................................................... reload tables
cr dump ....................................................... show in memory tables
cr addrt
cr rmrt
cr addcarrier
(prob, strip, rewrite_prefix, rewrite_suffix,...................
flags, mask and comment are optional arguments) ...............
cr rmcarrier
-- command 'rpid' - manage Remote-Party-ID (RPID)
rpid add
rpid rm
rpid show
-- command 'add|passwd|rm' - manage subscribers
add
passwd
rm
-- command 'add|dump|reload|rm|show' - manage trusted
trusted show ...................... show db content
trusted dump ...................... show cache content
trusted reload .................... reload db table into cache
trusted add
....................... add a new entry
....................... (from_pattern and tag are optional arguments)
trusted rm
-- command 'dispatcher' - manage dispatcher
* Examples: dispatcher addgw 1 sip:1.2.3.1:5050 1 'outbound gateway'
* dispatcher addgw 2 sip:1.2.3.4:5050 3 ''
* dispatcher rmgw 4
dispatcher show ..................... show dispatcher gateways
dispatcher reload ................... reload dispatcher gateways
dispatcher dump ..................... show in memory dispatcher gateways
dispatcher addgw
.......................... add gateway
dispatcher rmgw
-- command 'db' - database operations
db exec
db roexec
db run
db rorun
$id variable
db show
Modify kamailo config to make Contact attribute become aware (public ip address)
The client_nat_test/fix_contact had been moved to the nat_traversal module (they are locarted mediaproxy module before v1.3(include))
Just modify the route block inside the /usr/local/etc/kamailio/kamailo.cfg
if (is_method("REGISTER"))
{
# My adding part
if (client_nat_test("3")) {
fix_contact();
}
if (!save("location"))
sl_reply_error();
exit;
}
#Another you can add the nat_keepalive like following
if ((is_method("REGISTER") || is_method("SUBSCRIBE") || is_method("INVITE") && !has_totag()) && client_nat_test("3"))
{
nat_keepalive();
}
Reference: nat traversal module document (v1.4), some snapshots..
1.5.1. client_nat_test(type)
Check if the client is behind NAT. What tests are performed is specified by the type parameter which is an integer given by the sum of the numbers corresponsing to the tests that one wishes to perform. The numbers corresponding to individual tests are shown below:
*
1 - tests if client has a private IP address (as defined by RFC1918) in the Contact field of the SIP message.
*
2 - tests if client has contacted Kamailio from an address that is different from the one in the Via field. Both the IP and port are compared by this test.
*
4 - tests if client has a private IP address (as defined by RFC1918) in the top Via field of the SIP message.
For example calling client_nat_test("3") will perform test 1 and test 2 and return true if at least one succeeds, otherwise false.
This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE, FAILURE_ROUTE, BRANCH_ROUTE.
Example 1.6. Using the client_nat_test function
...
if (client_nat_test("3")) {
.....
}
...
1.5.2. fix_contact()
Will replace the IP and port in the Contact header with the IP and port the SIP message was received from. Usually called after a succesful call to client_nat_test(type)
This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE, BRANCH_ROUTE.
Example 1.7. Using the fix_contact function
...
if (client_nat_test("3")) {
fix_contact();
}
...
1.5.3. nat_keepalive()
Trigger keepalive functionality for the source address of the request. When called it only sets some internal flags, which will trigger later the addition of the endpoint to the keepalive list if a positive reply is generated/received (for REGISTER and SUBSCRIBE) or when the dialog is started/replied (for INVITEs). For this reason, it can be called early or late in the script. The only condition is to call it before replying to the request or before sending it to another proxy. If the request needs to be sent to another proxy, t_relay() must be used to be able to intercept replies via TM or dialog callbacks. If stateless forwarding is used, the keepalive functionality will not work. Also for outgoing INVITEs, record_route() should also be used to make sure the proxy that keeps the caller endpoint alive stays in the path. For multi-proxy setups, this function should always be called on the border proxies (the ones that received the request directly from the user agent). For more details about this function, see the Implementation subsection from the Keepalive functionality section.
This function can be used from REQUEST_ROUTE.
Example 1.8. Using the nat_keepalive function
...
if ((method=="REGISTER" || method=="SUBSCRIBE" ||
(method=="INVITE" && !has_totag())) && client_nat_test("3"))
{
nat_keepalive();
}
...
Just modify the route block inside the /usr/local/etc/kamailio/kamailo.cfg
if (is_method("REGISTER"))
{
# My adding part
if (client_nat_test("3")) {
fix_contact();
}
if (!save("location"))
sl_reply_error();
exit;
}
#Another you can add the nat_keepalive like following
if ((is_method("REGISTER") || is_method("SUBSCRIBE") || is_method("INVITE") && !has_totag()) && client_nat_test("3"))
{
nat_keepalive();
}
Reference: nat traversal module document (v1.4), some snapshots..
1.5.1. client_nat_test(type)
Check if the client is behind NAT. What tests are performed is specified by the type parameter which is an integer given by the sum of the numbers corresponsing to the tests that one wishes to perform. The numbers corresponding to individual tests are shown below:
*
1 - tests if client has a private IP address (as defined by RFC1918) in the Contact field of the SIP message.
*
2 - tests if client has contacted Kamailio from an address that is different from the one in the Via field. Both the IP and port are compared by this test.
*
4 - tests if client has a private IP address (as defined by RFC1918) in the top Via field of the SIP message.
For example calling client_nat_test("3") will perform test 1 and test 2 and return true if at least one succeeds, otherwise false.
This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE, FAILURE_ROUTE, BRANCH_ROUTE.
Example 1.6. Using the client_nat_test function
...
if (client_nat_test("3")) {
.....
}
...
1.5.2. fix_contact()
Will replace the IP and port in the Contact header with the IP and port the SIP message was received from. Usually called after a succesful call to client_nat_test(type)
This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE, BRANCH_ROUTE.
Example 1.7. Using the fix_contact function
...
if (client_nat_test("3")) {
fix_contact();
}
...
1.5.3. nat_keepalive()
Trigger keepalive functionality for the source address of the request. When called it only sets some internal flags, which will trigger later the addition of the endpoint to the keepalive list if a positive reply is generated/received (for REGISTER and SUBSCRIBE) or when the dialog is started/replied (for INVITEs). For this reason, it can be called early or late in the script. The only condition is to call it before replying to the request or before sending it to another proxy. If the request needs to be sent to another proxy, t_relay() must be used to be able to intercept replies via TM or dialog callbacks. If stateless forwarding is used, the keepalive functionality will not work. Also for outgoing INVITEs, record_route() should also be used to make sure the proxy that keeps the caller endpoint alive stays in the path. For multi-proxy setups, this function should always be called on the border proxies (the ones that received the request directly from the user agent). For more details about this function, see the Implementation subsection from the Keepalive functionality section.
This function can be used from REQUEST_ROUTE.
Example 1.8. Using the nat_keepalive function
...
if ((method=="REGISTER" || method=="SUBSCRIBE" ||
(method=="INVITE" && !has_totag())) && client_nat_test("3"))
{
nat_keepalive();
}
...
SIP attribute meaning
Contact is the location where UA can receive the message from the SIP Server (Registrar)
Contact:
Original from sip-ua the contact is private address, like the following describes
Contact:
After the openser processing, it will modify to the public one (e.g. 1.2.3.4)
Contact:;expires=240
Contact:
Original from sip-ua the contact is private address, like the following describes
Contact:
After the openser processing, it will modify to the public one (e.g. 1.2.3.4)
Contact:
2008年11月7日 星期五
How to Configure OpenSER: SIP Registar, SIP Proxy and Far-End NAT Traversal for Media
http://www.jeremy-mcnamara.com/2007/03/28/how-to-configure-openser-sip-registar-sip-proxy-and-far-end-nat-traversal-for-media/
OpenSER is a flexible, mature and stable SIP communications server. However unlike Asterisk, OpenSER requires extensive knowledge of the SIP Protocol and presumes nothing about your configuration or method of deployment.
I am basing this configuration on OpenSER v1.1.1 which is currently considered the most 'stable' version. Although v1.2 has been released, I am going to let others find and kill the lingering bugs before diving in myself.
This configuration will utilize the necessary modules and configuration directives of OpenSER to create a SIP Registrar, conditionally detect and route based on the dialed URI (digits). Plus we will overcome most NAT situations that many home and small business level deployments will encounter, using the MediaProxy option. (The other option being the older RTPProxy method)
The IP Address for this configuration will be 1.2.3.4, of course you will need to replace this with your own IP address. You also must setup an instance of MySQL and allow OpenSER to communicate with it (MySQL can very easily run on the same machine) I will use 'openser' as the username and 'valid_pass' as the password.
Once you have MySQL running you need to find the 'mysqldb.sh' script located in the scripts directory inside of the OpenSER v1.1.1 tarball.
This will create the 'openser' database, tables and required initial data.
proxy:# sh ./mysqldb.sh create
Next copy the file 'openserctl.mysql' from the scripts directory into /usr/local/lib/openser/openserctl/ directory
proxy:# cp openserctl.mysql /usr/local/lib/openser/openserctl/
The OpenSER make install process should do this for you, but as of v1.1.1 this did not happen for me.
Finally we can dive into the configuration of OpenSER. The default OpenSER configuration file is /usr/local/etc/openser/openser.cfg
Lets dive right in.
openser.cfg:
debug=3
fork=no
log_stderror=yes
Those three options set the stage for how OpenSER will act and function. By not forking and logging to STDERR one can run openser directly to gather information about an invalid configuration directive or perhaps why a specific module isn't loading as expected. However to start OpenSER using openserctl you will need to set fork=yes and log_stderror=no.
listen=1.2.3.4
alias=1.2.3.4
port=5060
The next 3 inform OpenSER which interface and tcp/dup ports to listen on. Pretty standard stuff, so far.
children=4
The children directive informs OpenSER of how many 'child' threads to keep around to process incoming messages. 4 seems to be a reasonable default for most systems. If you really start to scale up your OpenSER instance you will want to bump this value up, but remember each additional thread takes more memory so there is somewhat of a trade off - performance versus memory usage.
dns=no
rev_dns=no
Disabling the looking up up DNS is the most scalable option. Only enable the looking up of DNS if you really need it. In fact, I haven't found a really good reason for OpenSER to need to know about DNS names.
fifo="/tmp/openser_fifo"
fifo_db_url="mysql://openser:valid_pass@localhost/openser"
Those define the FIFOs that OpenSER will use to communicate on. Remember to provide valid database information.
loadmodule "/usr/local/lib/openser/modules/mysql.so"
loadmodule "/usr/local/lib/openser/modules/sl.so"
loadmodule "/usr/local/lib/openser/modules/tm.so"
loadmodule "/usr/local/lib/openser/modules/rr.so"
loadmodule "/usr/local/lib/openser/modules/maxfwd.so"
loadmodule "/usr/local/lib/openser/modules/usrloc.so"
loadmodule "/usr/local/lib/openser/modules/registrar.so"
loadmodule "/usr/local/lib/openser/modules/auth.so"
loadmodule "/usr/local/lib/openser/modules/auth_db.so"
loadmodule "/usr/local/lib/openser/modules/uri.so"
loadmodule "/usr/local/lib/openser/modules/uri_db.so"
loadmodule "/usr/local/lib/openser/modules/mediaproxy.so"
loadmodule "/usr/local/lib/openser/modules/nathelper.so"
loadmodule "/usr/local/lib/openser/modules/textops.so"
loadmodule "/usr/local/lib/openser/modules/domain.so"
loadmodule "/usr/local/lib/openser/modules/xlog.so"
loadmodule "/usr/local/lib/openser/modules/uac.so"
loadmodule "/usr/local/lib/openser/modules/speeddial.so"
loadmodule "/usr/local/lib/openser/modules/avpops.so"
The loading of the necessary modules is the meat and potatos for this configuration of OpenSER. I won't go into the specifics of each module here as the OpenSER team has done an excellent job of documenting the modules.
modparam("usrloc|auth_db|domain|speeddial|acc", "db_url", "mysql://openser:valid_pass@localhost/openser")
modparam("auth_db", "calculate_ha1", 1)
modparam("auth_db", "use_domain", 0)
modparam("domain", "db_mode", 1)
modparam("nathelper", "rtpproxy_disable", 1)
modparam("nathelper", "natping_interval", 60)
modparam("mediaproxy","natping_interval", 30)
modparam("mediaproxy","mediaproxy_socket", "/var/run/mediaproxy.sock")
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "use_domain", 0)
modparam("registrar", "default_expires", 60)
modparam("registrar", "min_expires", 30)
modparam("registrar", "nat_flag", 6)
modparam("registrar", "use_domain", 0)
modparam("rr", "enable_full_lr", 1)
modparam("auth", "rpid_suffix", ";party=calling;id-type=subscriber;screen=yes")
modparam("auth", "rpid_avp", "s:rpid")
Same goes for the modparams - The OpenSER team has documented these quite nicely. I will point out that we disable the rtpproxy, enable mediaproxy, enable write-through database mode for the user location, enable full loose routing support (to appease broken SIP User Agents), and setup a few 'flags' for logging, accounting and NAT.
Flags are a fun concept - Generally speaking a flag in OpenSER is like sticking a PostIT? Note on to specific SIP messages. Then later on in the 'route' processing we can check to see if specific notes have been left, making for much cleaner and simpler configuration.
Now we get into the "main" section of the configuration. In the OpenSER configuration everything is ran through a route block - meaning every SIP message gets processed thru a block of code named route (or you can think about it as a 'main' function, if you are a programmer)
route {
# Sanity Check
# ------------
if (msg:len > max_len) {
sl_send_reply("513", "Message Overflow");
return;;
};
if (!mf_process_maxfwd_header("10")) {
sl_send_reply("483", "Too Many Hops");
return;
};
The first two directives in our route block ensure we don't process a message that is too large (buffer overflow) or a message that we believe to be looping or otherwise confused.
# Record Route and NAT Preset
# --------------------
if (method == "INVITE" && client_nat_test("3")) {
# Must add valid IP address below
record_route_preset("1.2.3.4:5060;nat=yes");
} else if (method != "REGISTER") {
record_route();
};
The 'if' directive here checks to see if the current SIP message (in OpenSER terms this is called a 'method') is an INVITE and calls a function to determine if the entity sending the current SIP message is behind NAT or not. The '3′ instructs the function to test using the first two test methods, which are detailed on the NATHELPER module. If both of those conditions are true, we call a function that appends a small bit of information to the SIP message, which we will utilize later on in the route and its related blocks.
Then If the method is not a REGISTER, we setup the necessary SIP headers to ensure that further SIP messages come back through our SIP Proxy, by calling the record_route() function.
# Call Tear down
if (method=="BYE" || method=="CANCEL") {
end_media_session();
};
Then a call get hung up, we need to detect this to ensure our RTP MediaProxy gets informed to stop relaying audio. We also set our first flag. In this case we inform the accounting module that we would like to account for Bye and Cancel methods - meaning we want 'stop' Call Detail Records. Since we defined '1′ as our accounting flag, we simply pass a '1′ to the setflag() function - Pretty simple. The accounting module takes care of everything else for us.
# Loose Route
# -----------
if (loose_route()) {
if (has_totag() && (method == "INVITE" || method == "ACK")) {
if (client_nat_test("3") || search("^Route:.*;nat=yes")) {
setflag(6);
use_media_proxy();
};
};
route(1);
return;
};
This is a rather complex and powerful group of directives. Without getting into the gory details of the SIP Protocol, this block determines if we are using the 'loose routing' concept. If so, we need to check for NAT (possibly setting flag 6) and informing the MediaProxy we will need its services. Most often than not we will be using the loose routing conventions versus strict routing, as defined in RFC3261.
We also call route(1) - In our configuration this is what we call the 'default' route or the terminating route. We will document the implementation and usage of route(1) here in a few minutes.
# Call Type Processing
# --------------------
if (uri != myself) {
route(1);
return;
};
If the 'call' is no longer for this particular proxy - send the message on its way (via the 'default' route). This is sort of another sanity check for our simple configuration. However in more complex situations, examining the URI and properly routing the messages will become very important. Unfortunately those concepts are beyond the scope of this document.
if (uri == myself) {
if (method == "BYE") {
route(4);
return;
} else if (method == "CANCEL") {
route(4);
return;
} else if (method == "INVITE") {
route(3);
return;
} else if (method == "REGISTER") {
route(2);
return;
} else if (method == "NOTIFY") {
sl_send_reply("200", "Understood");
return;
} else if (method == "OPTIONS") {
sl_send_reply("200", "Got it");
return;
}
};
route(1);
}
Finally we reach the end of our route block. We test each SIP method type and send them to an appropriate 'route' blocks, with the exceptions being NOTIFY and OPTIONS methods. These two methods typcially get used (by Asterisk and many SIP Phones) to either assist in keeping firewall/NAT paths open and/or determining if the proxy is still allve or not. Asterisk can also calculate (qualify=1000) how long it takes for the message to be responded to, giving a sort of internet path health status. By processing the NOTIFY and OPTIONS methods here, we can avoid 404 and/or Too Many Hops message, which in themselves are not necessarily a bad thing, but could lead to confusion.
# Default Message Handling
# -----------------------
route[1] {
t_on_reply("1");
if (!t_relay()) {
if (method=="INVITE" || method=="ACK") {
end_media_session();
};
sl_reply_error();
};
}
Now we are down to the specific route blocks. As we discussed above, route[1] is the default message route. Here we ensure the SIP message gets properly responded to or sent on to its intended location, if the message is intended for another SIP Proxy.
We also check to see if we are processing an INVITE or ACK (to an invite) so we can inform the MediaProxy to end its session.
# REGISTER Message Handling
# -------------------------
route[2] {
if (!search("^Contact:\ +\*") && client_nat_test("7")) {
setflag(6);
fix_nated_register();
force_rport();
};
sl_send_reply("100", "Trying");
if (!www_authorize("","subscriber")) {
www_challenge("","0");
return;
};
if (!check_to()) {
sl_send_reply("401", "Unauthorized");
return;
};
consume_credentials();
if (!save("location")) {
sl_reply_error();
};
}
route[2] is our REGISTER method processing block. This is where we examine the credentials and determine if we are going to allow the SIP UA to register or not.
Notice the call to consume_credentials(), this is a very useful function that removes the unnecessary SIP headers that can contain sensitive authentication details.
# INVITE Message Handling
# ----------------------------------
route[3] {
# Test for nat, perhaps fix headers
if (client_nat_test("3")) {
setflag(7);
force_rport();
fix_nated_contact();
};
# 3 and 4 digits URIs are sent to our Asterisk PBX
if ((uri =~ "^sip:[0-9]{3}@.*") || (uri =~ "^sip:[0-9]{4}@.*")) {
rewritehost("pbx.valid.host.com:5060");
use_media_proxy();
route(1);
return;
};
# Any URI that begins with 1 plus 10 digits authenticate and pass on
# to our SIP Provider
if (uri =~ "^sip:1[0-9]{10}@.*") {
# Authenticate these calls
if (!proxy_authorize("","subscriber")) {
proxy_challenge("","0?);
return;
} else if (!check_from()) {
sl_send_reply("403?, "Use From ID");
return;
};
consume_credentials();
rewritehostport("proxy-1.nufone.net:5060?);
route(1);
return;
};
if (!lookup("location")) {
sl_send_reply("404?, "User Not Found, Sorry");
return;
};
# If NAT is previously detected, proxy
if (isflagset(6) || isflagset(7)) {
use_media_proxy();
};
route(1);
}
route[3] is our INVITE (or call setup) processing occurs. I have set this configuration up to match on 3 and 4 digit 'extensions' to send them to our Asterisk PBX. Then match on 1+10 digits to send off to our VoIP Provider, making sure to set the accounting flag (1) so we can get proper 'start' Call Detail Records.
# CANCEL and BYE Message Handling
# ----------------------------------
route[4] {
if (client_nat_test("3")) {
setflag(7);
force_rport();
fix_nated_contact();
};
end_media_session();
route(1);
}
route[4] is pretty straight forward - If its a CANCEL or BYE Message, ensure the message gets sent back to the proper location (ip:port). Then inform the MediaProxy to stop processing audio, since the call is ending.
onreply_route[1] {
if ((isflagset(6) || isflagset(7)) && (status=~"(180)|(183)|2[0-9][0-9]")) {
if (!search("^Content-Length:\ +0")) {
use_media_proxy();
};
};
if (client_nat_test("1")) {
fix_nated_contact();
};
}
The onreply_route[1] block gets called when we are expected to reply to a message (like a progress message or an OK). We ensure the MediaProxy gets used if NAT has been previously set or detected and the Contact SIP header is properly 'fixed' for NAT situations.
There you have it. This by far is NOT a comprehensive guide to configuring OpenSER, but I hope it gives you a deeper understanding of the power that we have available to us. Be sure to checkout the OpenSER Configuration Wizard, which provides an more complex configuration situations.
OpenSER is a magical and wonderful Open Source project that has and will continue to power many VoIP Providers and forward thinking enterprises for years to come.
I need your help. I am part of a team that is putting together OpenSER and Asterisk training material. If you have any questions or would like something specific covered, please take 2 minutes and fill out my OpenSER and Asterisk Training Questionnaire.
Related Posts:
VoIP Blog
OpenSER vs SER
Run Your Own SIP Server, Today!
SER/OpenSER Configuration Wizard
Multiple Via Headers in SIP messages
This entry was posted on Wednesday, March 28th, 2007 at 9:05 pm and is filed under How To, OpenSER, VoIP. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
6 Responses to "How to Configure OpenSER: SIP Registar, SIP Proxy and Far-End NAT Traversal for Media"
1. Configuration file for OpenSER v1.2.x to do VoIP NAT traversal using RTPProxy. - Voxilla VoIP Forum on June 28th, 2007 at 2:07 pm
[…] located. Send in the last few lines of that output if you are still confused. Also, check out my OpenSER How To. It may help you understand things a little bit […]
2. Run Your Own SIP Server, Today! by Jeremy McNamara on July 21st, 2007 at 11:27 pm
[…] Unlike Asterisk, OpenSER only deals with the SIP Signaling. This focus allows for a very high levels of scale and flexibility of configuration. Be sure to check out my OpenSER HowTo and check out the OpenSER Wizard, which generates very powerful configurations for multiple situations. […]
3. OpenSER vs SER by Jeremy McNamara on September 7th, 2007 at 10:59 pm
[…] there are two different variations of the same basic platform. I have previously published an OpenSER tutorial however there is another SIP server called […]
4. Blackeye1010 on November 27th, 2007 at 4:24 pm
Hello,
Your handling of sip methods, send BYE and CANCEL to a route block of theyr own. I'm using a config derived from the onsip site. At this link http://siprouter.onsip.org/doc/gettingstarted/ch08.html#mp_handle_cancel
is stated that:
"We now explicitly handle CANCEL messages. CANCEL messages can be safely processed with a simple call to t_relay() because SER will automatically match the CANCEL message to the original INVITE message. So here we just route the message to the default message handler."
…so i'm sending my BYES and CANCELS to the generalist route[1].
Would you say it's better or worse ? Or did you do that because Openser behaves diferent in this case ?
Regards
5. George on May 13th, 2008 at 6:58 pm
You keep talking about setting the accounting flag to '1′ but I don't see it anywhere
6. katiamong on June 1st, 2008 at 5:36 am
hi jeremy, i had a problem in forwarding INVITE message to asterisk.
i've tried with basic authentification with mysql anda forwarding it to asterisk if there a call to ext 701 (just example). i've always got message 'loop detected" and i try many solution it still happen.
Later i try using your configuration using media proxy, i've got problem with authetification (when sosftphone is registering the number, it says forbidden).
is there a solution how to make it works ?
i'm sorry to interupt you but you say in your blog that you need a lot of FAQ, and have no idea where i could post my question in your blog.
OpenSER is a flexible, mature and stable SIP communications server. However unlike Asterisk, OpenSER requires extensive knowledge of the SIP Protocol and presumes nothing about your configuration or method of deployment.
I am basing this configuration on OpenSER v1.1.1 which is currently considered the most 'stable' version. Although v1.2 has been released, I am going to let others find and kill the lingering bugs before diving in myself.
This configuration will utilize the necessary modules and configuration directives of OpenSER to create a SIP Registrar, conditionally detect and route based on the dialed URI (digits). Plus we will overcome most NAT situations that many home and small business level deployments will encounter, using the MediaProxy option. (The other option being the older RTPProxy method)
The IP Address for this configuration will be 1.2.3.4, of course you will need to replace this with your own IP address. You also must setup an instance of MySQL and allow OpenSER to communicate with it (MySQL can very easily run on the same machine) I will use 'openser' as the username and 'valid_pass' as the password.
Once you have MySQL running you need to find the 'mysqldb.sh' script located in the scripts directory inside of the OpenSER v1.1.1 tarball.
This will create the 'openser' database, tables and required initial data.
proxy:# sh ./mysqldb.sh create
Next copy the file 'openserctl.mysql' from the scripts directory into /usr/local/lib/openser/openserctl/ directory
proxy:# cp openserctl.mysql /usr/local/lib/openser/openserctl/
The OpenSER make install process should do this for you, but as of v1.1.1 this did not happen for me.
Finally we can dive into the configuration of OpenSER. The default OpenSER configuration file is /usr/local/etc/openser/openser.cfg
Lets dive right in.
openser.cfg:
debug=3
fork=no
log_stderror=yes
Those three options set the stage for how OpenSER will act and function. By not forking and logging to STDERR one can run openser directly to gather information about an invalid configuration directive or perhaps why a specific module isn't loading as expected. However to start OpenSER using openserctl you will need to set fork=yes and log_stderror=no.
listen=1.2.3.4
alias=1.2.3.4
port=5060
The next 3 inform OpenSER which interface and tcp/dup ports to listen on. Pretty standard stuff, so far.
children=4
The children directive informs OpenSER of how many 'child' threads to keep around to process incoming messages. 4 seems to be a reasonable default for most systems. If you really start to scale up your OpenSER instance you will want to bump this value up, but remember each additional thread takes more memory so there is somewhat of a trade off - performance versus memory usage.
dns=no
rev_dns=no
Disabling the looking up up DNS is the most scalable option. Only enable the looking up of DNS if you really need it. In fact, I haven't found a really good reason for OpenSER to need to know about DNS names.
fifo="/tmp/openser_fifo"
fifo_db_url="mysql://openser:valid_pass@localhost/openser"
Those define the FIFOs that OpenSER will use to communicate on. Remember to provide valid database information.
loadmodule "/usr/local/lib/openser/modules/mysql.so"
loadmodule "/usr/local/lib/openser/modules/sl.so"
loadmodule "/usr/local/lib/openser/modules/tm.so"
loadmodule "/usr/local/lib/openser/modules/rr.so"
loadmodule "/usr/local/lib/openser/modules/maxfwd.so"
loadmodule "/usr/local/lib/openser/modules/usrloc.so"
loadmodule "/usr/local/lib/openser/modules/registrar.so"
loadmodule "/usr/local/lib/openser/modules/auth.so"
loadmodule "/usr/local/lib/openser/modules/auth_db.so"
loadmodule "/usr/local/lib/openser/modules/uri.so"
loadmodule "/usr/local/lib/openser/modules/uri_db.so"
loadmodule "/usr/local/lib/openser/modules/mediaproxy.so"
loadmodule "/usr/local/lib/openser/modules/nathelper.so"
loadmodule "/usr/local/lib/openser/modules/textops.so"
loadmodule "/usr/local/lib/openser/modules/domain.so"
loadmodule "/usr/local/lib/openser/modules/xlog.so"
loadmodule "/usr/local/lib/openser/modules/uac.so"
loadmodule "/usr/local/lib/openser/modules/speeddial.so"
loadmodule "/usr/local/lib/openser/modules/avpops.so"
The loading of the necessary modules is the meat and potatos for this configuration of OpenSER. I won't go into the specifics of each module here as the OpenSER team has done an excellent job of documenting the modules.
modparam("usrloc|auth_db|domain|speeddial|acc", "db_url", "mysql://openser:valid_pass@localhost/openser")
modparam("auth_db", "calculate_ha1", 1)
modparam("auth_db", "use_domain", 0)
modparam("domain", "db_mode", 1)
modparam("nathelper", "rtpproxy_disable", 1)
modparam("nathelper", "natping_interval", 60)
modparam("mediaproxy","natping_interval", 30)
modparam("mediaproxy","mediaproxy_socket", "/var/run/mediaproxy.sock")
modparam("usrloc", "db_mode", 2)
modparam("usrloc", "use_domain", 0)
modparam("registrar", "default_expires", 60)
modparam("registrar", "min_expires", 30)
modparam("registrar", "nat_flag", 6)
modparam("registrar", "use_domain", 0)
modparam("rr", "enable_full_lr", 1)
modparam("auth", "rpid_suffix", ";party=calling;id-type=subscriber;screen=yes")
modparam("auth", "rpid_avp", "s:rpid")
Same goes for the modparams - The OpenSER team has documented these quite nicely. I will point out that we disable the rtpproxy, enable mediaproxy, enable write-through database mode for the user location, enable full loose routing support (to appease broken SIP User Agents), and setup a few 'flags' for logging, accounting and NAT.
Flags are a fun concept - Generally speaking a flag in OpenSER is like sticking a PostIT? Note on to specific SIP messages. Then later on in the 'route' processing we can check to see if specific notes have been left, making for much cleaner and simpler configuration.
Now we get into the "main" section of the configuration. In the OpenSER configuration everything is ran through a route block - meaning every SIP message gets processed thru a block of code named route (or you can think about it as a 'main' function, if you are a programmer)
route {
# Sanity Check
# ------------
if (msg:len > max_len) {
sl_send_reply("513", "Message Overflow");
return;;
};
if (!mf_process_maxfwd_header("10")) {
sl_send_reply("483", "Too Many Hops");
return;
};
The first two directives in our route block ensure we don't process a message that is too large (buffer overflow) or a message that we believe to be looping or otherwise confused.
# Record Route and NAT Preset
# --------------------
if (method == "INVITE" && client_nat_test("3")) {
# Must add valid IP address below
record_route_preset("1.2.3.4:5060;nat=yes");
} else if (method != "REGISTER") {
record_route();
};
The 'if' directive here checks to see if the current SIP message (in OpenSER terms this is called a 'method') is an INVITE and calls a function to determine if the entity sending the current SIP message is behind NAT or not. The '3′ instructs the function to test using the first two test methods, which are detailed on the NATHELPER module. If both of those conditions are true, we call a function that appends a small bit of information to the SIP message, which we will utilize later on in the route and its related blocks.
Then If the method is not a REGISTER, we setup the necessary SIP headers to ensure that further SIP messages come back through our SIP Proxy, by calling the record_route() function.
# Call Tear down
if (method=="BYE" || method=="CANCEL") {
end_media_session();
};
Then a call get hung up, we need to detect this to ensure our RTP MediaProxy gets informed to stop relaying audio. We also set our first flag. In this case we inform the accounting module that we would like to account for Bye and Cancel methods - meaning we want 'stop' Call Detail Records. Since we defined '1′ as our accounting flag, we simply pass a '1′ to the setflag() function - Pretty simple. The accounting module takes care of everything else for us.
# Loose Route
# -----------
if (loose_route()) {
if (has_totag() && (method == "INVITE" || method == "ACK")) {
if (client_nat_test("3") || search("^Route:.*;nat=yes")) {
setflag(6);
use_media_proxy();
};
};
route(1);
return;
};
This is a rather complex and powerful group of directives. Without getting into the gory details of the SIP Protocol, this block determines if we are using the 'loose routing' concept. If so, we need to check for NAT (possibly setting flag 6) and informing the MediaProxy we will need its services. Most often than not we will be using the loose routing conventions versus strict routing, as defined in RFC3261.
We also call route(1) - In our configuration this is what we call the 'default' route or the terminating route. We will document the implementation and usage of route(1) here in a few minutes.
# Call Type Processing
# --------------------
if (uri != myself) {
route(1);
return;
};
If the 'call' is no longer for this particular proxy - send the message on its way (via the 'default' route). This is sort of another sanity check for our simple configuration. However in more complex situations, examining the URI and properly routing the messages will become very important. Unfortunately those concepts are beyond the scope of this document.
if (uri == myself) {
if (method == "BYE") {
route(4);
return;
} else if (method == "CANCEL") {
route(4);
return;
} else if (method == "INVITE") {
route(3);
return;
} else if (method == "REGISTER") {
route(2);
return;
} else if (method == "NOTIFY") {
sl_send_reply("200", "Understood");
return;
} else if (method == "OPTIONS") {
sl_send_reply("200", "Got it");
return;
}
};
route(1);
}
Finally we reach the end of our route block. We test each SIP method type and send them to an appropriate 'route' blocks, with the exceptions being NOTIFY and OPTIONS methods. These two methods typcially get used (by Asterisk and many SIP Phones) to either assist in keeping firewall/NAT paths open and/or determining if the proxy is still allve or not. Asterisk can also calculate (qualify=1000) how long it takes for the message to be responded to, giving a sort of internet path health status. By processing the NOTIFY and OPTIONS methods here, we can avoid 404 and/or Too Many Hops message, which in themselves are not necessarily a bad thing, but could lead to confusion.
# Default Message Handling
# -----------------------
route[1] {
t_on_reply("1");
if (!t_relay()) {
if (method=="INVITE" || method=="ACK") {
end_media_session();
};
sl_reply_error();
};
}
Now we are down to the specific route blocks. As we discussed above, route[1] is the default message route. Here we ensure the SIP message gets properly responded to or sent on to its intended location, if the message is intended for another SIP Proxy.
We also check to see if we are processing an INVITE or ACK (to an invite) so we can inform the MediaProxy to end its session.
# REGISTER Message Handling
# -------------------------
route[2] {
if (!search("^Contact:\ +\*") && client_nat_test("7")) {
setflag(6);
fix_nated_register();
force_rport();
};
sl_send_reply("100", "Trying");
if (!www_authorize("","subscriber")) {
www_challenge("","0");
return;
};
if (!check_to()) {
sl_send_reply("401", "Unauthorized");
return;
};
consume_credentials();
if (!save("location")) {
sl_reply_error();
};
}
route[2] is our REGISTER method processing block. This is where we examine the credentials and determine if we are going to allow the SIP UA to register or not.
Notice the call to consume_credentials(), this is a very useful function that removes the unnecessary SIP headers that can contain sensitive authentication details.
# INVITE Message Handling
# ----------------------------------
route[3] {
# Test for nat, perhaps fix headers
if (client_nat_test("3")) {
setflag(7);
force_rport();
fix_nated_contact();
};
# 3 and 4 digits URIs are sent to our Asterisk PBX
if ((uri =~ "^sip:[0-9]{3}@.*") || (uri =~ "^sip:[0-9]{4}@.*")) {
rewritehost("pbx.valid.host.com:5060");
use_media_proxy();
route(1);
return;
};
# Any URI that begins with 1 plus 10 digits authenticate and pass on
# to our SIP Provider
if (uri =~ "^sip:1[0-9]{10}@.*") {
# Authenticate these calls
if (!proxy_authorize("","subscriber")) {
proxy_challenge("","0?);
return;
} else if (!check_from()) {
sl_send_reply("403?, "Use From ID");
return;
};
consume_credentials();
rewritehostport("proxy-1.nufone.net:5060?);
route(1);
return;
};
if (!lookup("location")) {
sl_send_reply("404?, "User Not Found, Sorry");
return;
};
# If NAT is previously detected, proxy
if (isflagset(6) || isflagset(7)) {
use_media_proxy();
};
route(1);
}
route[3] is our INVITE (or call setup) processing occurs. I have set this configuration up to match on 3 and 4 digit 'extensions' to send them to our Asterisk PBX. Then match on 1+10 digits to send off to our VoIP Provider, making sure to set the accounting flag (1) so we can get proper 'start' Call Detail Records.
# CANCEL and BYE Message Handling
# ----------------------------------
route[4] {
if (client_nat_test("3")) {
setflag(7);
force_rport();
fix_nated_contact();
};
end_media_session();
route(1);
}
route[4] is pretty straight forward - If its a CANCEL or BYE Message, ensure the message gets sent back to the proper location (ip:port). Then inform the MediaProxy to stop processing audio, since the call is ending.
onreply_route[1] {
if ((isflagset(6) || isflagset(7)) && (status=~"(180)|(183)|2[0-9][0-9]")) {
if (!search("^Content-Length:\ +0")) {
use_media_proxy();
};
};
if (client_nat_test("1")) {
fix_nated_contact();
};
}
The onreply_route[1] block gets called when we are expected to reply to a message (like a progress message or an OK). We ensure the MediaProxy gets used if NAT has been previously set or detected and the Contact SIP header is properly 'fixed' for NAT situations.
There you have it. This by far is NOT a comprehensive guide to configuring OpenSER, but I hope it gives you a deeper understanding of the power that we have available to us. Be sure to checkout the OpenSER Configuration Wizard, which provides an more complex configuration situations.
OpenSER is a magical and wonderful Open Source project that has and will continue to power many VoIP Providers and forward thinking enterprises for years to come.
I need your help. I am part of a team that is putting together OpenSER and Asterisk training material. If you have any questions or would like something specific covered, please take 2 minutes and fill out my OpenSER and Asterisk Training Questionnaire.
Related Posts:
VoIP Blog
OpenSER vs SER
Run Your Own SIP Server, Today!
SER/OpenSER Configuration Wizard
Multiple Via Headers in SIP messages
This entry was posted on Wednesday, March 28th, 2007 at 9:05 pm and is filed under How To, OpenSER, VoIP. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
6 Responses to "How to Configure OpenSER: SIP Registar, SIP Proxy and Far-End NAT Traversal for Media"
1. Configuration file for OpenSER v1.2.x to do VoIP NAT traversal using RTPProxy. - Voxilla VoIP Forum on June 28th, 2007 at 2:07 pm
[…] located. Send in the last few lines of that output if you are still confused. Also, check out my OpenSER How To. It may help you understand things a little bit […]
2. Run Your Own SIP Server, Today! by Jeremy McNamara on July 21st, 2007 at 11:27 pm
[…] Unlike Asterisk, OpenSER only deals with the SIP Signaling. This focus allows for a very high levels of scale and flexibility of configuration. Be sure to check out my OpenSER HowTo and check out the OpenSER Wizard, which generates very powerful configurations for multiple situations. […]
3. OpenSER vs SER by Jeremy McNamara on September 7th, 2007 at 10:59 pm
[…] there are two different variations of the same basic platform. I have previously published an OpenSER tutorial however there is another SIP server called […]
4. Blackeye1010 on November 27th, 2007 at 4:24 pm
Hello,
Your handling of sip methods, send BYE and CANCEL to a route block of theyr own. I'm using a config derived from the onsip site. At this link http://siprouter.onsip.org/doc/gettingstarted/ch08.html#mp_handle_cancel
is stated that:
"We now explicitly handle CANCEL messages. CANCEL messages can be safely processed with a simple call to t_relay() because SER will automatically match the CANCEL message to the original INVITE message. So here we just route the message to the default message handler."
…so i'm sending my BYES and CANCELS to the generalist route[1].
Would you say it's better or worse ? Or did you do that because Openser behaves diferent in this case ?
Regards
5. George on May 13th, 2008 at 6:58 pm
You keep talking about setting the accounting flag to '1′ but I don't see it anywhere
6. katiamong on June 1st, 2008 at 5:36 am
hi jeremy, i had a problem in forwarding INVITE message to asterisk.
i've tried with basic authentification with mysql anda forwarding it to asterisk if there a call to ext 701 (just example). i've always got message 'loop detected" and i try many solution it still happen.
Later i try using your configuration using media proxy, i've got problem with authetification (when sosftphone is registering the number, it says forbidden).
is there a solution how to make it works ?
i'm sorry to interupt you but you say in your blog that you need a lot of FAQ, and have no idea where i could post my question in your blog.
2008年11月5日 星期三
httrack - web site copier
Some examples
httrack www.someweb.com/bob/
mirror site www.someweb.com/bob/ and only this site
==> The above command is work for me. But for some web site (http://samba.org/ftp/unpacked/junkcode/), it will not able to get the data from that site. Until now I don't know the real reason.
httrack www.someweb.com/bob/ www.anothertest.com/mike/ +*.com/*.jpg -mime:application/*
mirror the two sites together (with shared links) and accept any .jpg files on .com
sites
httrack www.someweb.com/bob/bobby.html +* -r6
means get all files starting from bobby.html, with 6 link-depth, and possibility of
going everywhere on the web
httrack www.someweb.com/bob/bobby.html --spider -P proxy.myhost.com:8080
runs the spider on www.someweb.com/bob/bobby.html using a proxy
httrack --update
updates a mirror in the current folder
httrack
will bring you to the interactive mode
httrack --continue
continues a mirror in the current folder
httrack www.someweb.com/bob/
mirror site www.someweb.com/bob/ and only this site
==> The above command is work for me. But for some web site (http://samba.org/ftp/unpacked/junkcode/), it will not able to get the data from that site. Until now I don't know the real reason.
httrack www.someweb.com/bob/ www.anothertest.com/mike/ +*.com/*.jpg -mime:application/*
mirror the two sites together (with shared links) and accept any .jpg files on .com
sites
httrack www.someweb.com/bob/bobby.html +* -r6
means get all files starting from bobby.html, with 6 link-depth, and possibility of
going everywhere on the web
httrack www.someweb.com/bob/bobby.html --spider -P proxy.myhost.com:8080
runs the spider on www.someweb.com/bob/bobby.html using a proxy
httrack --update
updates a mirror in the current folder
httrack
will bring you to the interactive mode
httrack --continue
continues a mirror in the current folder
Some hints in the C programming
1. Before using the character strings (either char array or malloc buffer), it is better to clean up all the buffer to zero (using memset(buf, 0, bufsize) or bzero function)
Question:
Sometimes we use the following declaration
char *mystr;
Then we can assign a string to the variable "mystr", like this. Is it valid to use this method?
mystr = "hello world";
Question:
Sometimes we use the following declaration
char *mystr;
Then we can assign a string to the variable "mystr", like this. Is it valid to use this method?
mystr = "hello world";
2008年10月29日 星期三
osip2 experience
Some state definitions in the osip2 header file
ICT_XXXX means invite client transaction
IST_XXXX means invite server transaction
NICT_XXXX means NON-invite client transaction
NIST_XXXX means NON-invite server transaction
defined in the libosip2-3.1.0/include/osip2/osip.h
ICT_XXXX means invite client transaction
IST_XXXX means invite server transaction
NICT_XXXX means NON-invite client transaction
NIST_XXXX means NON-invite server transaction
defined in the libosip2-3.1.0/include/osip2/osip.h
2008年10月27日 星期一
ubuntu apt-get experience
apt-get install screen
The following description is a snapshot from the http://linuxhelp.blogspot.com/2005/12/concise-apt-get-dpkg-primer-for-new.html
December 13, 2005
A Concise apt-get / dpkg primer for new Debian users
Debian is one of the earliest Linux distribution around. It caught the public's fancy because of the ease of installing and uninstalling applications on it. When many other linux distributions were bogged down in dependency hell, Debian users were shielded from these problems owing to Debian's superior package handling capablities using apt-get.
All Linux distributions which claim their roots in the Debian distribution use this versatile package manager. For the uninitiated, Debian uses the deb package format for bundling together files belonging to an application. You can look at it as something like a setup installer (Eg: Installshield) in windows counterpart.
Here I will explain how to go about using this package handling utility to get the results that you desire.
The first step needed to use apt-get to your advantage is including the necessary repositories. Repositories are merely collections of softwares which are stored in a public location on the internet. By including the web address of these repositories, you are directing apt-get to search these locations for the desired software. You use the /etc/apt/sources.list file to list the addresses of the repositories. It takes the following format:
deb [web address] [distribution name][maincontribnon-free]
For example, in Ubuntu a debian based distribution, it could be something like this:
deb http://in.archive.ubuntu.com/ubuntu breezy main restrcted
You can add any repository you like. apt-get.org contains an excellent collection of repositories to suite all tastes.
Once you have set the repositories, the next step is to sync the local software database with the database on the repositories. This will cache a copy of the list of all the remotely available softwares to your machine. This is achieved by running the following command:
# apt-get update
An advantage of this is you now have the power to search for a particular program to see if it is available for your version of distribution using the apt-cache command. And you don't need a net connection to do this. For example,
# apt-cache search baseutils
... will tell me if the package baseutils is available in the repository or not by searching the locally cached copy of the database.
Once you have figured that the package (in our case baseutils) is available, then installing it is as simple as running the following command:
# apt-get install baseutils
The real power of apt-get is realised now. If the baseutils package depends on the availability of a version of the library say, "xyz1.5.6.so". Then apt-get will download the library (or package containing the library) from the net and install it before installing baseutils package. This is known as automatic dependency resolution.
And removing a package is as simple as running the command:
# apt-get remove baseutils
Get statistics about the packages available in the repositories by running the command :
# apt-cache stats
Total package names : 22502 (900k)
Normal packages: 17632
Pure virtual packages: 281
Single virtual packages: 1048
Mixed virtual packages: 172
Missing: 3369
...
To upgrade all the softwares on your system to the latest versions, do the following:
# apt-get upgrade
And finally the king of them all - upgrading the whole distribution to a new version can be done with the command:
# apt-get dist-upgrade
Saving valuable hard disk space
Each time you install an application using apt-get, the package is actually cached in a location on your hard disk. It is usually stored in the location /var/cache/apt/archives/ . Over a period of time, all the cached packages will eat up your valuable hard disk space. You can clear the cache and release hard disk space by using the following command:
# apt-get clean
You could also use autoclean where in, only those packages in the cache which are found useless or partially complete are deleted.
# apt-get autoclean
dpkg - The low level Package management utility
As I said earlier, Debian based distributions use the Deb package format. Usually normal users like you and me are shielded from handling individual deb packages. But if you fall into a situation where you have to install a deb package you use the dpkg utility.
Lets assume I have a deb package called gedit-2.12.1.deb and I want to install it on my machine. I do it using the following command:
# dpkg -i gedit-2.12.1.deb
To remove an installed package, run the command:
# dpkg -r gedit
The main thing to note above is I have used only the name of the program and not the version number while removing the software.
You may also use the --purge (-P) flag for removing software.
# dpkg -P gedit
This will remove gedit along with all its configuration files. Where as -r (--remove) does not delete the configuration files.
Now lets say I do not want to actually install a package but want to see the contents of a Deb package. This can be achieved using the -c flag:
# dpkg -c gedit-2.12.1.deb
To get more information about a package like the authors name,the year in which it was compiled and a short description of its use, you use the -I flag:
# dpkg -I gedit-2.12.1.deb
You can even use wild cards to list the packages on your machine. For example, to see all the gcc packages on your machine, do the following:
# dpkg -l gcc*
Desired=Unknown/Install/Remove/Purge/Hold
Status=Not/Installed/Config-files/Unpacked/Failed-config/.
/ Err?=(none)/Hold/Reinst-required/X=both-problems
/ Name Version Description
+++-===============-==============-========================
ii gcc 4.0.1-3 The GNU C compiler
ii gcc-3.3-base 3.3.6-8ubuntu1 The GNU Compiler Colletio
un gcc-3.5 none (no description available)
un gcc-3.5-base none (no description available)
un gcc-3.5-doc none (no description available)
ii gcc-4.0 4.0.1-4ubuntu9 The GNU C compiler
...
In the above listing, the first 'i' denotes desired state which is install. The second 'i' denotes the actual state ie gcc is installed. The third column gives the error problems if any. The fourth, fifth and sixth column gives the name, version and description of the packages respectively. And gcc-3.5 is not installed on my machine. So the status is given as 'un' which is unknown not-installed.
To check if an individual package is installed, you use the status -s flag:
# dpkg -s gedit
Two days back, I installed beagle (a real time search tool based on Mono) on my machine. But I didn't have a clue about the location of the files as well as what files were installed along with beagle. That was when I used the -L option to get a list of all the files installed by the beagle package.
# dpkg -L beagle
Even better, you can combine the above command with grep to get a listing of all the html documentation of beagle.
# dpkg -L beagle | grep html$
These are just a small sample of the options you can use with dpkg utility. To know more about this tool, check its man page.
If you are alergic to excessive command line activities, then you may also use dselect which is a curses based menu driven front-end to the low level dpkg utility.
GUI front-ends for apt-get
* Synaptic
* Aptitude
The following description is a snapshot from the http://linuxhelp.blogspot.com/2005/12/concise-apt-get-dpkg-primer-for-new.html
December 13, 2005
A Concise apt-get / dpkg primer for new Debian users
Debian is one of the earliest Linux distribution around. It caught the public's fancy because of the ease of installing and uninstalling applications on it. When many other linux distributions were bogged down in dependency hell, Debian users were shielded from these problems owing to Debian's superior package handling capablities using apt-get.
All Linux distributions which claim their roots in the Debian distribution use this versatile package manager. For the uninitiated, Debian uses the deb package format for bundling together files belonging to an application. You can look at it as something like a setup installer (Eg: Installshield) in windows counterpart.
Here I will explain how to go about using this package handling utility to get the results that you desire.
The first step needed to use apt-get to your advantage is including the necessary repositories. Repositories are merely collections of softwares which are stored in a public location on the internet. By including the web address of these repositories, you are directing apt-get to search these locations for the desired software. You use the /etc/apt/sources.list file to list the addresses of the repositories. It takes the following format:
deb [web address] [distribution name][maincontribnon-free]
For example, in Ubuntu a debian based distribution, it could be something like this:
deb http://in.archive.ubuntu.com/ubuntu breezy main restrcted
You can add any repository you like. apt-get.org contains an excellent collection of repositories to suite all tastes.
Once you have set the repositories, the next step is to sync the local software database with the database on the repositories. This will cache a copy of the list of all the remotely available softwares to your machine. This is achieved by running the following command:
# apt-get update
An advantage of this is you now have the power to search for a particular program to see if it is available for your version of distribution using the apt-cache command. And you don't need a net connection to do this. For example,
# apt-cache search baseutils
... will tell me if the package baseutils is available in the repository or not by searching the locally cached copy of the database.
Once you have figured that the package (in our case baseutils) is available, then installing it is as simple as running the following command:
# apt-get install baseutils
The real power of apt-get is realised now. If the baseutils package depends on the availability of a version of the library say, "xyz1.5.6.so". Then apt-get will download the library (or package containing the library) from the net and install it before installing baseutils package. This is known as automatic dependency resolution.
And removing a package is as simple as running the command:
# apt-get remove baseutils
Get statistics about the packages available in the repositories by running the command :
# apt-cache stats
Total package names : 22502 (900k)
Normal packages: 17632
Pure virtual packages: 281
Single virtual packages: 1048
Mixed virtual packages: 172
Missing: 3369
...
To upgrade all the softwares on your system to the latest versions, do the following:
# apt-get upgrade
And finally the king of them all - upgrading the whole distribution to a new version can be done with the command:
# apt-get dist-upgrade
Saving valuable hard disk space
Each time you install an application using apt-get, the package is actually cached in a location on your hard disk. It is usually stored in the location /var/cache/apt/archives/ . Over a period of time, all the cached packages will eat up your valuable hard disk space. You can clear the cache and release hard disk space by using the following command:
# apt-get clean
You could also use autoclean where in, only those packages in the cache which are found useless or partially complete are deleted.
# apt-get autoclean
dpkg - The low level Package management utility
As I said earlier, Debian based distributions use the Deb package format. Usually normal users like you and me are shielded from handling individual deb packages. But if you fall into a situation where you have to install a deb package you use the dpkg utility.
Lets assume I have a deb package called gedit-2.12.1.deb and I want to install it on my machine. I do it using the following command:
# dpkg -i gedit-2.12.1.deb
To remove an installed package, run the command:
# dpkg -r gedit
The main thing to note above is I have used only the name of the program and not the version number while removing the software.
You may also use the --purge (-P) flag for removing software.
# dpkg -P gedit
This will remove gedit along with all its configuration files. Where as -r (--remove) does not delete the configuration files.
Now lets say I do not want to actually install a package but want to see the contents of a Deb package. This can be achieved using the -c flag:
# dpkg -c gedit-2.12.1.deb
To get more information about a package like the authors name,the year in which it was compiled and a short description of its use, you use the -I flag:
# dpkg -I gedit-2.12.1.deb
You can even use wild cards to list the packages on your machine. For example, to see all the gcc packages on your machine, do the following:
# dpkg -l gcc*
Desired=Unknown/Install/Remove/Purge/Hold
Status=Not/Installed/Config-files/Unpacked/Failed-config/.
/ Err?=(none)/Hold/Reinst-required/X=both-problems
/ Name Version Description
+++-===============-==============-========================
ii gcc 4.0.1-3 The GNU C compiler
ii gcc-3.3-base 3.3.6-8ubuntu1 The GNU Compiler Colletio
un gcc-3.5 none (no description available)
un gcc-3.5-base none (no description available)
un gcc-3.5-doc none (no description available)
ii gcc-4.0 4.0.1-4ubuntu9 The GNU C compiler
...
In the above listing, the first 'i' denotes desired state which is install. The second 'i' denotes the actual state ie gcc is installed. The third column gives the error problems if any. The fourth, fifth and sixth column gives the name, version and description of the packages respectively. And gcc-3.5 is not installed on my machine. So the status is given as 'un' which is unknown not-installed.
To check if an individual package is installed, you use the status -s flag:
# dpkg -s gedit
Two days back, I installed beagle (a real time search tool based on Mono) on my machine. But I didn't have a clue about the location of the files as well as what files were installed along with beagle. That was when I used the -L option to get a list of all the files installed by the beagle package.
# dpkg -L beagle
Even better, you can combine the above command with grep to get a listing of all the html documentation of beagle.
# dpkg -L beagle | grep html$
These are just a small sample of the options you can use with dpkg utility. To know more about this tool, check its man page.
If you are alergic to excessive command line activities, then you may also use dselect which is a curses based menu driven front-end to the low level dpkg utility.
GUI front-ends for apt-get
* Synaptic
* Aptitude
Apache, MySQL service
# mysqld and apached is the same usage in the linux
/usr/local/mysql/bin/mysqld_safe &
/usr/local/apache2/bin/apachectl start
Q. How to disable httpd port 443 listening?
In the /etc/httpd/conf.d folder, there are many modules configure file. One config file named ssl.conf is the https setting.
Mark the line of "Listen 443", and then the httpd launch will not bind on the port 443 anymore.
PS. /etc/httpd/conf/httpd.conf is the main apache config file.
/etc/httpd/conf.d/ contains all the apache sub-module config files.
/sbin/ipnat -f /etc/ipnat.conf
/sbin/sysctl -w net.inet.ip.forwarding=1
/usr/share/denyhosts/daemon-control start
/usr/local/mysql/bin/mysqld_safe &
/usr/local/apache2/bin/apachectl start
Q. How to disable httpd port 443 listening?
In the /etc/httpd/conf.d folder, there are many modules configure file. One config file named ssl.conf is the https setting.
Mark the line of "Listen 443", and then the httpd launch will not bind on the port 443 anymore.
PS. /etc/httpd/conf/httpd.conf is the main apache config file.
/etc/httpd/conf.d/ contains all the apache sub-module config files.
/sbin/ipnat -f /etc/ipnat.conf
/sbin/sysctl -w net.inet.ip.forwarding=1
/usr/share/denyhosts/daemon-control start
Using vMware server unable to connect to the server with vmware server daemon
We can using netstat to show that vMware server daemon is listening on the port 904 using tcp
# netstat -anp | grep 904 tcp 0 0 :::904 :::* LISTEN 3786/xinetd
And we also disable the iptables daemon
# service iptables stop
Flush all the iptables rules in the filter table
# iptables -F
But we still can not use vmware server to login into the destined machine. Finally I found the problem is caused by the routing settings. Because the destined machine has a interface configued the ip-address the same subnet with the connecting machine. So the returned packet will route the different path with the forward path. So the vMware will show the destination is unreachable like the attached diagram.
2008年10月22日 星期三
Linux Command - awk experience
Example1.
#/etc/links1 檔案內容
WAN1=rtk0
WAN2=rtk1
LAN1=rtk2
DMZ1=rtk3
DMZ2=rtk4
WAN3=rtk5
計算總行數
# gawk '/WAN/{count++}END{print count}' /etc/links1
3
Example2.
用"="號來區隔出資料變數，並存在var陣列中
# gawk 'BEGIN{split("DMZ=rtk3",var,"=");print var[2]}'
rtk3
awk預設的分隔符號為space(空白鍵)，在這邊將其預設欄位分隔符號改為"="，再取出該檔案的第2欄資料
# gawk -F'=' '{ print $2 }' /etc/links
rtk0
rtk1
rtk2
rtk3
rtk4
rtk5
Example3.
將/etc/ifconfig.rtk0的第三欄資料印出
#gawk '{print $3}' /etc/ifconfig.rtk0`
Show the line if the packets count is non zero
iptables -t mangle -L -v -n | awk '$1!=0 {print $0}'
Chain Chain1 (1 references)
pkts bytes target prot opt in out source destination
1 78 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto nbns
2 99 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto rdp
4 226 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto telnet
$ ls -la | awk '($1 = /-rw-r--r--/ ){print $9}'
aa.gif
Example.
file aaa 的內容
Total run 5 times
ftp_0.pat allchar-sets 0 printchar-sets 0
ftp_1.pat allchar-sets 0 printchar-sets 0
ftp_2.pat allchar-sets 0 printchar-sets 0
ftp_3.pat allchar-sets 5 printchar-sets 0
msn_0.pat allchar-sets 1 printchar-sets 1
msn_1.pat allchar-sets 0 printchar-sets 0
msn_2.pat allchar-sets 0 printchar-sets 0
msn_3.pat allchar-sets 0 printchar-sets 6
msn_4.pat allchar-sets 0 printchar-sets 0
msn_5.pat allchar-sets 0 printchar-sets 0
取出第三欄或是第五欄內容不為0 的records
# cat aaa | awk 'NR>1 && ($3!=0 || $5!=0){print $0}'
ftp_3.pat allchar-sets 5 printchar-sets 0
msn_0.pat allchar-sets 1 printchar-sets 1
msn_3.pat allchar-sets 0 printchar-sets 6
每隔3秒鐘, 印/proc/slabinfo 中的某些資料
# while [ 1 ]; do cat /proc/slabinfo | awk '/\/{print $2, $3}' | tail -1; sleep 3; done
/flash/etc/system.conf 內容
PASSTHROUGHPKT=5000
PASSTHROUGHBYTE=1000000
#RANDOMRATE=40
# awk -F "=" '/^PASSTHROUGHBYTE/{print $2}' /flash/etc/system.conf
1000000
VER= $(shell awk -F"[:|@]" '{ print $$3 }' CVS/Root)
$ cat CVS/Root
:ext:vincent@192.168.17.190:/home/cvsroot
$ cat CVS/Root | awk -F"[:|@]" '{ print $3 }' // 這裡awk -F 的separator 利用 [:|@] or 的符號
my_cvs_username
HOST_IP=$(shell ifconfig eth0 | grep "inet addr" | awk -F: '{print $$2}' | awk '{print $$1}')
$ ifconfig eth0
Warning: cannot open /proc/net/dev (Permission denied). Limited output.
eth0 Link encap:Ethernet HWaddr 00:0E:A6:44:ED:A7
inet addr:192.168.211.18 Bcast:192.168.211.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:209 Memory:fbffc000-0
STARTTIME= $(shell LC_ALL=C date )
LC_ALL=C 的目的是啥? :(
這裡我比較不懂的地方有
1. 在Makefile 裡面, 需要利用shell 來做shell command execution, 格式如下
$(shell command-list)
command list 就是原先在shell 理面執行的commands
2. 在Makefile 裡面的引數都要改成 $$1, $$2, $$3 (尤其是在shell command 當中), 原因是什麼, 我現在還不是很清楚
ls | awk '{print "echo filename:"$1" ; tcpdump -nv host 207.46.2.152 -r "$1}' | sh
ls | xargs -n 1 tcpdump -nv host 207.46.2.152 -r
ls -cr --sort=t | head -1 | xargs -n 1 md5sum
ls -lacr --sort=t | head -2 | awk 'NR!=1{print $8}'
InstantScan-10-prophet.bin
[vincent@CMF tftpboot]$ ls -lacr --sort=t | head -2 | awk 'NR!=1{print $8}' | md5sum // 這時處理的不是檔案, 是"字串"
eb1c42d6d509de2d02c6d426a6b74934 -
[vincent@CMF tftpboot]$ md5sum InstantScan-10-prophet.bin
c98bbabbd0d22cdc15b37d9759b78ba4 InstantScan-10-prophet.bin
ls -ur --sort=t // sort by access time, 由oldest time to the newest time (sorting descending)
ls -cr --sort=t // sort by last modification time, 由oldest time to the newest time (sorting descending)
ls -u --sort=t // sort by access time, 由newest time to the oldest time (sorting ascending)
ls -c --sort=t // sort by last modification time, 由newest time to the oldest time (sorting ascending)
// 測試 ok
ls -lcr --sort=t | head -100 | awk 'NR!=1{print $8}' | sudo xargs -n 1 rm -f
Some awk simple command
# more /etc/ifconfig.rtk0 | awk '{print $1}'
192.168.17.205
# gawk '{print $2}' /etc/ifconfig.rtk0
192.168.17.173
# cat /etc/ifconfig.rtk0 | gawk '{print $2}'
192.168.17.173
# more /etc/ifconfig.rtk0 | gawk '{print $2}'
192.168.17.173
定時顯示snortinline PID & time value
#!/bin/sh
while (true)
do
ps aux | grep snort_inline | grep -v grep | awk '{printf "%s ", $1}'; date | awk '{print $4}'
sleep 5;
done
結果
29205 11:28:59
29205 11:29:04
29205 11:29:09
29205 11:29:14
29205 11:29:19
29205 11:29:24
29205 11:29:29
29205 11:29:34
29205 11:29:39
29205 11:29:44
while [ 1 ]; do ps aux | grep snort | grep -v grep | awk '{print $1}'; sleep 20; done
#/etc/links1 檔案內容
WAN1=rtk0
WAN2=rtk1
LAN1=rtk2
DMZ1=rtk3
DMZ2=rtk4
WAN3=rtk5
計算總行數
# gawk '/WAN/{count++}END{print count}' /etc/links1
3
Example2.
用"="號來區隔出資料變數，並存在var陣列中
# gawk 'BEGIN{split("DMZ=rtk3",var,"=");print var[2]}'
rtk3
awk預設的分隔符號為space(空白鍵)，在這邊將其預設欄位分隔符號改為"="，再取出該檔案的第2欄資料
# gawk -F'=' '{ print $2 }' /etc/links
rtk0
rtk1
rtk2
rtk3
rtk4
rtk5
Example3.
將/etc/ifconfig.rtk0的第三欄資料印出
#gawk '{print $3}' /etc/ifconfig.rtk0`
Show the line if the packets count is non zero
iptables -t mangle -L -v -n | awk '$1!=0 {print $0}'
Chain Chain1 (1 references)
pkts bytes target prot opt in out source destination
1 78 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto nbns
2 99 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto rdp
4 226 RETURN all -- * * 0.0.0.0/0 0.0.0.0/0 LAYER7 l7proto telnet
$ ls -la | awk '($1 = /-rw-r--r--/ ){print $9}'
aa.gif
Example.
file aaa 的內容
Total run 5 times
ftp_0.pat allchar-sets 0 printchar-sets 0
ftp_1.pat allchar-sets 0 printchar-sets 0
ftp_2.pat allchar-sets 0 printchar-sets 0
ftp_3.pat allchar-sets 5 printchar-sets 0
msn_0.pat allchar-sets 1 printchar-sets 1
msn_1.pat allchar-sets 0 printchar-sets 0
msn_2.pat allchar-sets 0 printchar-sets 0
msn_3.pat allchar-sets 0 printchar-sets 6
msn_4.pat allchar-sets 0 printchar-sets 0
msn_5.pat allchar-sets 0 printchar-sets 0
取出第三欄或是第五欄內容不為0 的records
# cat aaa | awk 'NR>1 && ($3!=0 || $5!=0){print $0}'
ftp_3.pat allchar-sets 5 printchar-sets 0
msn_0.pat allchar-sets 1 printchar-sets 1
msn_3.pat allchar-sets 0 printchar-sets 6
每隔3秒鐘, 印/proc/slabinfo 中的某些資料
# while [ 1 ]; do cat /proc/slabinfo | awk '/\
/flash/etc/system.conf 內容
PASSTHROUGHPKT=5000
PASSTHROUGHBYTE=1000000
#RANDOMRATE=40
# awk -F "=" '/^PASSTHROUGHBYTE/{print $2}' /flash/etc/system.conf
1000000
VER= $(shell awk -F"[:|@]" '{ print $$3 }' CVS/Root)
$ cat CVS/Root
:ext:vincent@192.168.17.190:/home/cvsroot
$ cat CVS/Root | awk -F"[:|@]" '{ print $3 }' // 這裡awk -F 的separator 利用 [:|@] or 的符號
my_cvs_username
HOST_IP=$(shell ifconfig eth0 | grep "inet addr" | awk -F: '{print $$2}' | awk '{print $$1}')
$ ifconfig eth0
Warning: cannot open /proc/net/dev (Permission denied). Limited output.
eth0 Link encap:Ethernet HWaddr 00:0E:A6:44:ED:A7
inet addr:192.168.211.18 Bcast:192.168.211.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:209 Memory:fbffc000-0
STARTTIME= $(shell LC_ALL=C date )
LC_ALL=C 的目的是啥? :(
這裡我比較不懂的地方有
1. 在Makefile 裡面, 需要利用shell 來做shell command execution, 格式如下
$(shell command-list)
command list 就是原先在shell 理面執行的commands
2. 在Makefile 裡面的引數都要改成 $$1, $$2, $$3 (尤其是在shell command 當中), 原因是什麼, 我現在還不是很清楚
ls | awk '{print "echo filename:"$1" ; tcpdump -nv host 207.46.2.152 -r "$1}' | sh
ls | xargs -n 1 tcpdump -nv host 207.46.2.152 -r
ls -cr --sort=t | head -1 | xargs -n 1 md5sum
ls -lacr --sort=t | head -2 | awk 'NR!=1{print $8}'
InstantScan-10-prophet.bin
[vincent@CMF tftpboot]$ ls -lacr --sort=t | head -2 | awk 'NR!=1{print $8}' | md5sum // 這時處理的不是檔案, 是"字串"
eb1c42d6d509de2d02c6d426a6b74934 -
[vincent@CMF tftpboot]$ md5sum InstantScan-10-prophet.bin
c98bbabbd0d22cdc15b37d9759b78ba4 InstantScan-10-prophet.bin
ls -ur --sort=t // sort by access time, 由oldest time to the newest time (sorting descending)
ls -cr --sort=t // sort by last modification time, 由oldest time to the newest time (sorting descending)
ls -u --sort=t // sort by access time, 由newest time to the oldest time (sorting ascending)
ls -c --sort=t // sort by last modification time, 由newest time to the oldest time (sorting ascending)
// 測試 ok
ls -lcr --sort=t | head -100 | awk 'NR!=1{print $8}' | sudo xargs -n 1 rm -f
Some awk simple command
# more /etc/ifconfig.rtk0 | awk '{print $1}'
192.168.17.205
# gawk '{print $2}' /etc/ifconfig.rtk0
192.168.17.173
# cat /etc/ifconfig.rtk0 | gawk '{print $2}'
192.168.17.173
# more /etc/ifconfig.rtk0 | gawk '{print $2}'
192.168.17.173
定時顯示snortinline PID & time value
#!/bin/sh
while (true)
do
ps aux | grep snort_inline | grep -v grep | awk '{printf "%s ", $1}'; date | awk '{print $4}'
sleep 5;
done
結果
29205 11:28:59
29205 11:29:04
29205 11:29:09
29205 11:29:14
29205 11:29:19
29205 11:29:24
29205 11:29:29
29205 11:29:34
29205 11:29:39
29205 11:29:44
while [ 1 ]; do ps aux | grep snort | grep -v grep | awk '{print $1}'; sleep 20; done
2008年10月20日 星期一
Boot linux in the single mode
Boot linux in the single mode
http://www.cyberciti.biz/faq/grub-boot-into-single-user-mode/
(2) Select the kernel
(3) Press the e key to edit the entry
(4) Select second line (the line starting with the word kernel)
(5) Press the e key to edit kernel entry so that you can append single user mode
(6) Append the letter S (or word Single) to the end of the (kernel) line
(7) Press ENTER key
(8) Now press the b key to boot the Linux kernel into single user mode
(9) When prompted give root password and you be allowed to login into single user mode.
http://www.cyberciti.biz/tips/howto-recovering-grub-boot-loader-password.html
In Centos 5.4, this method seems not work. 20100305 testing
http://www.cyberciti.biz/faq/grub-boot-into-single-user-mode/
(2) Select the kernel
(3) Press the e key to edit the entry
(4) Select second line (the line starting with the word kernel)
(5) Press the e key to edit kernel entry so that you can append single user mode
(6) Append the letter S (or word Single) to the end of the (kernel) line
(7) Press ENTER key
(8) Now press the b key to boot the Linux kernel into single user mode
(9) When prompted give root password and you be allowed to login into single user mode.
http://www.cyberciti.biz/tips/howto-recovering-grub-boot-loader-password.html
In Centos 5.4, this method seems not work. 20100305 testing
2008年10月15日 星期三
crontab format
# +---------------- minute (0 - 59)
# | +------------- hour (0 - 23)
# | | +---------- day of month (1 - 31)
# | | | +------- month (1 - 12)
# | | | | +---- day of week (0 - 6) (Sunday=0 or 7)
# | | | | |
* * * * * command to be executed
// List the crontab settings
# crontab -l
// Edit the crontab settings
# crontab -e
# For example, every 5 minutes to run the /root/bin/prog1
*/5 * * * * /root/bin/prog1
# every Monday 06:00 AM backup the files
0 6 * * 1 /usr/local/sbin/backup.sh
# Each two days period on 06:00 AM backup the files
0 6 */2 * * /usr/local/sbin/cvsbackup.sh
# | +------------- hour (0 - 23)
# | | +---------- day of month (1 - 31)
# | | | +------- month (1 - 12)
# | | | | +---- day of week (0 - 6) (Sunday=0 or 7)
# | | | | |
* * * * * command to be executed
// List the crontab settings
# crontab -l
// Edit the crontab settings
# crontab -e
# For example, every 5 minutes to run the /root/bin/prog1
*/5 * * * * /root/bin/prog1
# every Monday 06:00 AM backup the files
0 6 * * 1 /usr/local/sbin/backup.sh
# Each two days period on 06:00 AM backup the files
0 6 */2 * * /usr/local/sbin/cvsbackup.sh
Common useful mysql commands
use root account to login to database "mysql"
# mysql -u root mysql
Show all fields of a selected table
mysql> desc user;
mysql> select User,Host from mysql.user;
Some on-line help
mysql> help contents;
You asked for help about help category: "Contents"
For more information, type 'help- ', where
- is one of the following
categories:
Account Management
Administration
Data Definition
Data Manipulation
Data Types
Functions
Functions and Modifiers for Use with GROUP BY
Geographic Features
Language Structure
Storage Engines
Stored Routines
Table Maintenance
Transactions
Triggers
mysql> Update user SET Insert_priv='y',Update_priv='y' where user='openser'; Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
# Before update the value. Show the record value of specified field in a indicated record (user)
mysql> Select User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Reload_priv From user WHERE user='openser';
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| User | Select_priv | Insert_priv | Update_priv | Delete_priv | Create_priv | Drop_priv | Reload_priv |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| openser | Y | Y | Y | N | N | N | N |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
1 row in set (0.00 sec)
# Update the value of specified field in a indicated record (user)
mysql> Update user SET Delete_priv='y',Create_priv='y',Drop_priv='y',Reload_priv='y' where user='openser'; Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
# After update the value. Show the record value of specified field in a indicated record (user)
mysql> Select User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Reload_priv From user WHERE user='openser';
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| User | Select_priv | Insert_priv | Update_priv | Delete_priv | Create_priv | Drop_priv | Reload_priv |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| openser | Y | Y | Y | Y | Y | Y | Y |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
1 row in set (0.00 sec)
# mysql -u root mysql
Show all fields of a selected table
mysql> desc user;
mysql> select User,Host from mysql.user;
Some on-line help
mysql> help contents;
You asked for help about help category: "Contents"
For more information, type 'help
categories:
Account Management
Administration
Data Definition
Data Manipulation
Data Types
Functions
Functions and Modifiers for Use with GROUP BY
Geographic Features
Language Structure
Storage Engines
Stored Routines
Table Maintenance
Transactions
Triggers
mysql> Update user SET Insert_priv='y',Update_priv='y' where user='openser'; Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
# Before update the value. Show the record value of specified field in a indicated record (user)
mysql> Select User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Reload_priv From user WHERE user='openser';
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| User | Select_priv | Insert_priv | Update_priv | Delete_priv | Create_priv | Drop_priv | Reload_priv |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| openser | Y | Y | Y | N | N | N | N |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
1 row in set (0.00 sec)
# Update the value of specified field in a indicated record (user)
mysql> Update user SET Delete_priv='y',Create_priv='y',Drop_priv='y',Reload_priv='y' where user='openser'; Query OK, 1 row affected (0.00 sec)
Rows matched: 1 Changed: 1 Warnings: 0
# After update the value. Show the record value of specified field in a indicated record (user)
mysql> Select User,Select_priv,Insert_priv,Update_priv,Delete_priv,Create_priv,Drop_priv,Reload_priv From user WHERE user='openser';
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| User | Select_priv | Insert_priv | Update_priv | Delete_priv | Create_priv | Drop_priv | Reload_priv |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
| openser | Y | Y | Y | Y | Y | Y | Y |
+---------+-------------+-------------+-------------+-------------+-------------+-----------+-------------+
1 row in set (0.00 sec)
Linux Command - sed
Example1.
#/etc/links1 檔案內容
WAN1=rtk0
WAN2=rtk1
LAN1=rtk2
DMZ1=rtk3
DMZ2=rtk4
WAN3=rtk5
# sed -n '3p' /etc/links1
LAN1=rtk2 -->只印出第3行
# sed '5p' /etc/links1
WAN1=rtk0
WAN2=rtk1
LAN1=rtk2
DMZ1=rtk3
DMZ2=rtk4
DMZ2=rtk4 --> 印出第5行
WAN3=rtk5
./Orig file
/usr/home/vincent/EP/target/vendors/D-Link/DFL-1500/www/help/help_files/ad_bm.html
/usr/home/vincent/EP/target/vendors/D-Link/DFL-1500/www/help/help_files/ad_bm_action.html
// 經過執行sed 之後
sed 's/EP\/target\/vendors\/D-Link\/DFL-1500\/www\/help/Help\/Current/g' ./Orig > ./Comp
./Comp file
/usr/home/vincent/Help/Current/help_files/ad_bm.html
/usr/home/vincent/Help/Current/help_files/ad_bm_action.html
研究如何將 "所有的檔案路徑移除，只留下檔案名稱"
Enterprise$ cat x
/abc/123
/abc/der/123
/usr/home/vincent/Help/Current/help_files/ad_bm_action.html
# sed 's/\/([a-zA-Z0-9_]* \ / )*//g' ./x ==> 目前利用這個方法無法work
Enterprise$ sed 's/\/.*\///g' ./x ==> 這個方法是work的，但是太簡單了，需要再想過更好的方法：)
123
123
ad_bm_action.html
.config
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.10
# Tue Nov 29 11:18:26 2005
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_BROKEN_ON_SMP=y
# sed -ne 's/^\([A-Z0-9_]*\)=\(.*\)$/#define \1 \2/p' < .config > ./output
output
#define CONFIG_X86 y
#define CONFIG_MMU y
#define CONFIG_UID16 y
#define CONFIG_GENERIC_ISA_DMA y
#define CONFIG_GENERIC_IOMAP y
[略]
The separate token can be either ; or /
# echo --datadir=/var/lib/mysql | sed -e 's;--datadir=;;'
/var/lib/mysql
# echo --datadir=/var/lib/mysql | sed -e 's/--datadir=//'
/var/lib/mysql
We can use sign "#" as the separater, the example is a snapshot of the kamailio Makefile install target
# sed -e "s#/usr/.*lib/kamailio/modules/#/usr/local/lib/kamailio/modules/#g" < etc/kamailio.cfg
# make original file as the filename add suffix name ".old"
sed -i.old -e 's%ftp://ftp.gnu.org/gnu/gcc/releases/gcc-%http://ftp.gnu.org/gnu/gcc/gcc-%' -e 's/gdb //' make/gcc-uclibc-3.3.mk
#/etc/links1 檔案內容
WAN1=rtk0
WAN2=rtk1
LAN1=rtk2
DMZ1=rtk3
DMZ2=rtk4
WAN3=rtk5
# sed -n '3p' /etc/links1
LAN1=rtk2 -->只印出第3行
# sed '5p' /etc/links1
WAN1=rtk0
WAN2=rtk1
LAN1=rtk2
DMZ1=rtk3
DMZ2=rtk4
DMZ2=rtk4 --> 印出第5行
WAN3=rtk5
./Orig file
/usr/home/vincent/EP/target/vendors/D-Link/DFL-1500/www/help/help_files/ad_bm.html
/usr/home/vincent/EP/target/vendors/D-Link/DFL-1500/www/help/help_files/ad_bm_action.html
// 經過執行sed 之後
sed 's/EP\/target\/vendors\/D-Link\/DFL-1500\/www\/help/Help\/Current/g' ./Orig > ./Comp
./Comp file
/usr/home/vincent/Help/Current/help_files/ad_bm.html
/usr/home/vincent/Help/Current/help_files/ad_bm_action.html
研究如何將 "所有的檔案路徑移除，只留下檔案名稱"
Enterprise$ cat x
/abc/123
/abc/der/123
/usr/home/vincent/Help/Current/help_files/ad_bm_action.html
# sed 's/\/([a-zA-Z0-9_]* \ / )*//g' ./x ==> 目前利用這個方法無法work
Enterprise$ sed 's/\/.*\///g' ./x ==> 這個方法是work的，但是太簡單了，需要再想過更好的方法：)
123
123
ad_bm_action.html
.config
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.10
# Tue Nov 29 11:18:26 2005
#
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_UID16=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_CLEAN_COMPILE=y
CONFIG_BROKEN_ON_SMP=y
# sed -ne 's/^\([A-Z0-9_]*\)=\(.*\)$/#define \1 \2/p' < .config > ./output
output
#define CONFIG_X86 y
#define CONFIG_MMU y
#define CONFIG_UID16 y
#define CONFIG_GENERIC_ISA_DMA y
#define CONFIG_GENERIC_IOMAP y
[略]
The separate token can be either ; or /
# echo --datadir=/var/lib/mysql | sed -e 's;--datadir=;;'
/var/lib/mysql
# echo --datadir=/var/lib/mysql | sed -e 's/--datadir=//'
/var/lib/mysql
We can use sign "#" as the separater, the example is a snapshot of the kamailio Makefile install target
# sed -e "s#/usr/.*lib/kamailio/modules/#/usr/local/lib/kamailio/modules/#g" < etc/kamailio.cfg
# make original file as the filename add suffix name ".old"
sed -i.old -e 's%ftp://ftp.gnu.org/gnu/gcc/releases/gcc-%http://ftp.gnu.org/gnu/gcc/gcc-%' -e 's/gdb //' make/gcc-uclibc-3.3.mk
2008年10月14日 星期二
Linux Command - Some useful tools
Show the file type ?
# type ulimit
ulimit is a shell builtin
# type vi
vi is aliased to `vim'
# type mysql
mysql is /usr/bin/mysql
# type mysqld_safe
mysqld_safe is /usr/bin/mysqld_safe
Question:
The "type" command is similar to the "file" command. What is the difference?
Answer:
The "file" command is used to determine the file type of a specified file.
And "type" command is used to determine the type of executable program. And "type" will also point the entire path of the specified program in the current file system.
# type ulimit
ulimit is a shell builtin
# type vi
vi is aliased to `vim'
# type mysql
mysql is /usr/bin/mysql
# type mysqld_safe
mysqld_safe is /usr/bin/mysqld_safe
Question:
The "type" command is similar to the "file" command. What is the difference?
Answer:
The "file" command is used to determine the file type of a specified file.
And "type" command is used to determine the type of executable program. And "type" will also point the entire path of the specified program in the current file system.
Setup mysql
http://dev.mysql.com/doc/refman/5.0/en/access-denied.html
1. setup grant table for system to do the access control, run the following command to create the table(in the mysql folder) which will located in the the mysql installation directory (e.g. /var/lob/mysql)
/usr/bin/mysql_install_db
/usr/local/bin/mysql_install_db --user=mysql
2. run the mysqld server, main starting script of mysql
/usr/bin/mysqld_safe &
3. ps aux | grep mysql
mysql 7634 0.7 0.8 135872 17516 pts/11 Sl 14:11 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
Generally, mysqld is statred with the following arguments
/usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
4. Stop the mysqld
Either use kill to terminate the mysql or use mysqladmin
# /usr/bin/mysqladmin -u root shutdown
5. Trouble shooting
read the error log of mysql "/var/log/mysqld.log"
5.1. We must create the user "mysql" manually (Doesn't mysql offer standard procedure to create the "mysql" user)
# useradd -m mysql
5.2. Refer to the following information, mysqld need to setup the process id but the directory is not yet created. So we need to create it manually and set its user/group as mysql.
[ERROR] /usr/local/libexec/mysqld: Can't create/write to file '/var/run/mysqld/mysqld.pid' (Errcode: 2)
081117 10:24:23 [ERROR] Can't start server: can't create PID file: No such file or directory
# mkdir /var/run/mysqld
chown mysql:mysql /var/run/mysqld/
default path of mysql file
the path of server daemon
==> /usr/libexec/mysqld
config file
==> /etc/my.cnf
default error log
==> err_log=/var/log/mysqld.log
default pid file
==> pid_file=/var/run/mysqld/mysqld.pid
data directory (mysql database repository?)
DATADIR=/var/lib/mysql
mysql use unix domain socket? so the specified unix socket file is in the following path..(just guesting.)
mysql_unix_port=/var/lib/mysql/mysql.sock
inside the mysqld_safe
default directory
MY_BASEDIR_VERSION=/usr
ledir=/usr/libexec
DATADIR=/var/lib/mysql
MYSQL_HOME=/usr
The my_print_defaults is only used in the mysql project?
man my_print_defaults
my_print_defaults - display options from option files
/usr/bin/my_print_defaults --loose-verbose mysqld server
One question, I can't kill the mysqld process. Either using kill -9 or kill -0. It is still alive. Just the process id is change only (since the process is killed but restored by the other mysql daemon)
# ps auxwww | grep mysql
mysql 10454 0.8 0.8 135872 17516 pts/11 Sl 17:35 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
# killall -9 mysqld
# ps auxwww | grep mysql
mysql 10482 0.7 0.8 135872 17516 pts/11 Sl 17:36 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
1. setup grant table for system to do the access control, run the following command to create the table(in the mysql folder) which will located in the the mysql installation directory (e.g. /var/lob/mysql)
/usr/bin/mysql_install_db
/usr/local/bin/mysql_install_db --user=mysql
2. run the mysqld server, main starting script of mysql
/usr/bin/mysqld_safe &
3. ps aux | grep mysql
mysql 7634 0.7 0.8 135872 17516 pts/11 Sl 14:11 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
Generally, mysqld is statred with the following arguments
/usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
4. Stop the mysqld
Either use kill to terminate the mysql or use mysqladmin
# /usr/bin/mysqladmin -u root shutdown
5. Trouble shooting
read the error log of mysql "/var/log/mysqld.log"
5.1. We must create the user "mysql" manually (Doesn't mysql offer standard procedure to create the "mysql" user)
# useradd -m mysql
5.2. Refer to the following information, mysqld need to setup the process id but the directory is not yet created. So we need to create it manually and set its user/group as mysql.
[ERROR] /usr/local/libexec/mysqld: Can't create/write to file '/var/run/mysqld/mysqld.pid' (Errcode: 2)
081117 10:24:23 [ERROR] Can't start server: can't create PID file: No such file or directory
# mkdir /var/run/mysqld
chown mysql:mysql /var/run/mysqld/
default path of mysql file
the path of server daemon
==> /usr/libexec/mysqld
config file
==> /etc/my.cnf
default error log
==> err_log=/var/log/mysqld.log
default pid file
==> pid_file=/var/run/mysqld/mysqld.pid
data directory (mysql database repository?)
DATADIR=/var/lib/mysql
mysql use unix domain socket? so the specified unix socket file is in the following path..(just guesting.)
mysql_unix_port=/var/lib/mysql/mysql.sock
inside the mysqld_safe
default directory
MY_BASEDIR_VERSION=/usr
ledir=/usr/libexec
DATADIR=/var/lib/mysql
MYSQL_HOME=/usr
The my_print_defaults is only used in the mysql project?
man my_print_defaults
my_print_defaults - display options from option files
/usr/bin/my_print_defaults --loose-verbose mysqld server
One question, I can't kill the mysqld process. Either using kill -9 or kill -0. It is still alive. Just the process id is change only (since the process is killed but restored by the other mysql daemon)
# ps auxwww | grep mysql
mysql 10454 0.8 0.8 135872 17516 pts/11 Sl 17:35 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
# killall -9 mysqld
# ps auxwww | grep mysql
mysql 10482 0.7 0.8 135872 17516 pts/11 Sl 17:36 0:00 /usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql --pid-file=/var/run/mysqld/mysqld.pid --skip-external-locking --socket=/var/lib/mysql/mysql.sock
2008年10月12日 星期日
Configure kamailio experience
First you need to setup the mysql if you want to use the db_mysql module
Setup mysql via yum
# yum install mysql.i386 ==> include mysql client software
# yum install mysql-server.i386 ==> include mysqld mysqld_safe.. server program
We need to create the mysql related tables. Before we create the mysql table. We need to specify the db-engine type in the kamctlrc config file.
/usr/local/etc/kamailio/kamctlrc
DBENGINE=MYSQL
Then use the kamailio database script(kamdbctl) to create the related table
/usr/sbin/kamdbctl create
Run the kamailio program
./kamailio -D -ddddddddd
How to make kamailio TLS work?
* First you must compile the kamailio-1.4.0-tls_src.tar.gz source tarball with the TLS=1 set in the Makefile
* Second you must make sure the "fork = yes" in the kamailio config file
Refer to the document in the TLS tutorial
http://www.kamailio.org/docs/tls-devel.html#TLS-EXAMPLE
1.7. OpenSER with TLS - script example
IMPORTANT: The TLS support is based on TCP, and for allowing OpenSER to use TCP, it must be started in multi-process mode. So, there is a must to have the "fork" parameter set to "yes":
NOTE: Since the TLS engine is quite memory consuming, increase the used memory by the run time parameter "-m" (see OpenSER -h for more details).
* fork = yes ==> The most important part
Setup mysql via yum
# yum install mysql.i386 ==> include mysql client software
# yum install mysql-server.i386 ==> include mysqld mysqld_safe.. server program
We need to create the mysql related tables. Before we create the mysql table. We need to specify the db-engine type in the kamctlrc config file.
/usr/local/etc/kamailio/kamctlrc
DBENGINE=MYSQL
Then use the kamailio database script(kamdbctl) to create the related table
/usr/sbin/kamdbctl create
Run the kamailio program
./kamailio -D -ddddddddd
How to make kamailio TLS work?
* First you must compile the kamailio-1.4.0-tls_src.tar.gz source tarball with the TLS=1 set in the Makefile
* Second you must make sure the "fork = yes" in the kamailio config file
Refer to the document in the TLS tutorial
http://www.kamailio.org/docs/tls-devel.html#TLS-EXAMPLE
1.7. OpenSER with TLS - script example
IMPORTANT: The TLS support is based on TCP, and for allowing OpenSER to use TCP, it must be started in multi-process mode. So, there is a must to have the "fork" parameter set to "yes":
NOTE: Since the TLS engine is quite memory consuming, increase the used memory by the run time parameter "-m" (see OpenSER -h for more details).
* fork = yes ==> The most important part
Configure bugzilla experience
利用checksetup.pl 設定 bugzilla root e-mail address, 盡可能設定reachable e-mail address
開設帳號, 可以利用user 自行login 來create new account 的方式 (New Account), bugzilla 發bug 之後, 會自動送出e-mail 到負責bug的e-mail 信箱中
必要的設定,
Parameters-->
Required Settings -->
User Authentication -->
Email ==> Can't find this option :(
Users use bugzilla system need to register account first. Or the bugzilla will reject any accessing actions for the not authenticated user.
Parameters-->
Required Settings -->
User Authentication -->
requirelogin (Set on, original off)
Only the bugzilla administrator can create user account. Disable the user self register the bugzilla system.
Parameters-->
Required Settings -->
User Authentication -->
createemailregexp (set "" blank, original .* allow all e-mails)
If we disable the feature of user self creating account, then the bugzilla administrator must create user account for the new coming user.
The original descriptions of the bugzilla system for the "createemailregexp" field
This defines the regexp to use for email addresses that are permitted to self-register using a 'New Account' feature. The default (.*) permits any account matching the emailregexp to be created. If this parameter is left blank, no users will be permitted to create their own accounts and all accounts will have to be created by an administrator.
開設帳號, 可以利用user 自行login 來create new account 的方式 (New Account), bugzilla 發bug 之後, 會自動送出e-mail 到負責bug的e-mail 信箱中
必要的設定,
Parameters-->
Required Settings -->
User Authentication -->
Email ==> Can't find this option :(
Users use bugzilla system need to register account first. Or the bugzilla will reject any accessing actions for the not authenticated user.
Parameters-->
Required Settings -->
User Authentication -->
requirelogin (Set on, original off)
Only the bugzilla administrator can create user account. Disable the user self register the bugzilla system.
Parameters-->
Required Settings -->
User Authentication -->
createemailregexp (set "" blank, original .* allow all e-mails)
If we disable the feature of user self creating account, then the bugzilla administrator must create user account for the new coming user.
The original descriptions of the bugzilla system for the "createemailregexp" field
This defines the regexp to use for email addresses that are permitted to self-register using a 'New Account' feature. The default (.*) permits any account matching the emailregexp to be created. If this parameter is left blank, no users will be permitted to create their own accounts and all accounts will have to be created by an administrator.
2008年10月8日 星期三
Linux Command - tcpdump/ethereal/wireshark
show the packets with specified length
# tcpdump -nvi eth0 port ssh and greater 1500
//tcpdump filter which tcp header src port isn't 3389 and destination port isn't 3389 too.
tcpdump -n -N -i eth1 tcp[0:2] != 3389 and tcp[2:2] != 3389
tcpdump -n -N -i eth1 tcp[0:2] = 445 or tcp[2:2] = 445
tcpdump -n -N -i eth1 'tcp[0:2]!=161 and tcp[2:2]!=161 and tcp[0:2]!=445 and tcp[2:2]!=445 and tcp[0:2]!=139 and tcp[2:2]!=139'
//下面兩個例子都是要求tcpdump去capture src/dest port > 1024 並且不等於3389port的connection
tcpdump -n -N -i eth1 'tcp[0:2] & 0xfc00!=0 and tcp[2:2] & 0xfc00!=0 and tcp[0:2]!=3389 and tcp[2:2]!=3389'
tcpdump -n -N -i eth1 'tcp[0:2]>1024 and tcp[2:2]>1024 and tcp[0:2]!=3389 and tcp[2:2]!=3389'
將tcpdump 的內容寫到檔案，以供tcpreplay 或 ethereal 來寫
tcpdump -n -i br0 ip host 170.116.11.94 and port 80 -w file
看某一個特定subnet
tcpdump -n -i br0 ip net 192.168.200.0/24
TCP Flags
URG ACK PSH RST SYN FIN
## tcp[13]==17 --> FIN, ACK
## tcp[13]==2 --> SYN
## tcp[13]==18 --> SYN, ACK
## tcp[13]==4 --> RST
# tcpdump -n -i br0 host 192.168.200.1 and tcp[13]==17 or tcp[13]=2
將tcpdump 的內容寫到檔案，以供tcpreplay 或 ethereal 來寫
tcpdump -n -i br0 ip host 170.116.11.94 and port 80 -w file
關於tcpdump 抓檔時, 所設定的-s snaplength 過小時, tcpreplay 在replay traffic 時, 會有的處理方式
tcpdump -s snaplen
tcpreplay -u
can be
pad -- pad the end of the packet with zeros
trunc -- re-adjusting the length in the IP header
-u or untruncate
When a packet is truncated in the capture file because the snaplen was too small, this option will pad the end of the packet with zeros, or truncate (trunc) it by re-adjusting the length in the IP header. The trunc option will only alter IPv4 packets, all others will be sent unmodified.
tcpreplay -R
tcpreplay -r, ex.
# tcpreplay -r 30
tcpreplay -m 0.1
Some early experience
//使用前須先執行，經過測試好像不用執行這行指令也可以work的樣子，不是很確定
//將eth0 interface設定為promisc mode，亦即網路上所有流經本地端的封包，都會被parsing
#ifconfig eth0 promisc
//將interface eth的promisc mode disable
#ifconfig eth0 -promisc
//不要使用promiscuous mode的情形下來節取packet，尚在測試中，不是很確定
#tcpdump -p
//dump source ip address 為 143.158.11.94的packet
#tcpdump src host 143.158.11.94
//dump source ip address & destination ipaddress為143.158.11.96的packet
#tcpdump host 143.158.11.96
//dump source port & destination port為80的packet
#tcpdump port 80
//dump destination network為143.158.11.0/24 的packet
# tcpdump dst net 143.158.11.0/24
# tcpdump -nvi eth2 ip net 172.16.0.0/16
>> dump a network wioth specified netmask
#./tcpdump -i wan1 -s 0 net 198.145.245.0 mask 255.255.255.0 -w aaa.pcap
>> or like this format
#./tcpdump -i wan1 -s 0 net 198.145.245.0/24 -w aaa.pcap
promiscuous mode 是指 ethernet card 接收全部 packet 的一種模式，正常的情況, ethernet card 是用 mode 3.
ethernet card 的receive mode 有:
01h turn off receiver
02h receive only packes sent to this interface
03h mode 2 plus broadcast packets
04h mode 3 plus limited multicast packets
05h mode 3 plus all multicast packets
06h all packets(promiscuous mode)
07h raw mode for serial line only(v1.10+)
你可能執行類似 tcpdump 的網路監聽程式, 才會有上述的 error mesg
網卡可以置一種模式叫混雜模式（promiscuous），在這種模式下工作的網卡能夠接收到一切通過它的數據，而不管實際上數 據的目的地址是不是他。這實際上就是我們SNIFF工作的基本原理：讓網卡接收一切他所能接收的數據。
Q. How to set network card in the promiscuous mode?
A.
# ifconfig eth0 promisc
// Be sure to replace "eth0" with your own network interface in case it's "wlan0" or something else.
// To remove promiscuous mode, type:
# ifconfig eth0 -promisc
ethereal/wireshark Analyzes => Display Filters can input the filtering rule to search the specified packets
e.g.
* tcp.flags.reset == 1
* tcp.len >= 1500
* frame.number==24333 (each packet is called a frame in the ethereal/wireshark, so we can use frame as the display filter component
* tcp.analysis.flags (use wireshark tcp analysis result)
* tcp.analysis.lost_segment
* tcp.analysis.retransmission
* tcp.analysis.fast_retransmission
* cdp.checksum_bad==1 || edp.checksum_bad==1 || ip.checksum_bad==1 || tcp.checksum_bad==1 || udp.checksum_bad==1
Advanced wireshark search tips
include:
Display filter:
e.g.
frame contains fe:3d:dd:36
or other display filtering rules
Hex value:
fe:3d:dd:36 or fe3ddd36
String:
"babala" (guessing usage...)
ipv6 filtering rules in the wireshark
icmpv6
ipv6.src
ipv6.dst
ipv6.addr == ff02::1:2
We can setup the coloring rule in the wireshark to separate the different packets type in the following method.
Select View -> Coloring Rules
Setup the coloring rules to display different color according the filtering rules.
wireshark will sniff all the packets destinated the Network Card buffer. And print the packets in the raw socket format.
We can disable the TCP checksum verification by the following steps:
1. Select Edit->Preferences
2. Select protocols -> TCP from the left frame of current window
3. Disable the option of "Validate the TCP checksum if possible"
Then the checksum error of TCP packets will be ignored by the wireshark.
TCP principle
* TCP often send an ACK packet while received a data packet, the acknowlegement number is the received packet sequence number plus received packet payload length.
* Sometimes TCP will send an Cumulative ACK if a preACK is not yet sent out because the data of sender is quickly enough (often less than 500 ms)
(According the RFC documents: TCP ACK generation[RFC 1122, RFC 2581])
* TCP will send a duplicated-ACK while it receive a miss-order packet (miss one packet and receive the afterward packet, wireshark call "TCP Previous segment lost").
* After receive a out-of-order packet (missing the previous packets, wireshark label "TCP Previous segment lost"), the wireshark will label all the following received packet "TCP Retransmission" until the packet sequence of previous received out-of-order packet.
* In wireshark, after receiving a packet labeled "
Some different tcp analysis in the wireshark
Dup ACK (Due to miss a packet of a next sequence number, this is labeled a "TCP Previous segment lost" in the wireshark)
Retransmission (Due to happen the specified RTT timeout and no ack received)
Fast Retransmission (Due to receive 3 ack packet with the same acknowledgement number) TCP Previous segment lost (Due to miss a packet which should in the next order, but receive a wrong order packet)
Refer the Fast Retransmission description as following:
http://en.wikipedia.org/wiki/Fast_retransmit
Fast Retransmit is an enhancement to TCP which reduces the time a sender waits before retransmitting a lost segment.
A TCP sender uses timers to recognize lost segments. If an acknowledgement is not received for a particular segment within a specified time (a function of the estimated Round-trip delay time), the sender will assume the segment was lost in the network, and will retransmit the segment.
The fast retransmit enhancement works as follows: if a TCP sender receives three duplicate acknowledgements with the same acknowledge number (that is, a total of four acknowledgements with the same acknowledgement number), the sender can be reasonably confident that the segment with the next higher sequence number was dropped, and will not arrive out of order. The sender will then retransmit the packet that was presumed dropped before waiting for its timeout.
# tcpdump -nvi eth0 port ssh and greater 1500
//tcpdump filter which tcp header src port isn't 3389 and destination port isn't 3389 too.
tcpdump -n -N -i eth1 tcp[0:2] != 3389 and tcp[2:2] != 3389
tcpdump -n -N -i eth1 tcp[0:2] = 445 or tcp[2:2] = 445
tcpdump -n -N -i eth1 'tcp[0:2]!=161 and tcp[2:2]!=161 and tcp[0:2]!=445 and tcp[2:2]!=445 and tcp[0:2]!=139 and tcp[2:2]!=139'
//下面兩個例子都是要求tcpdump去capture src/dest port > 1024 並且不等於3389port的connection
tcpdump -n -N -i eth1 'tcp[0:2] & 0xfc00!=0 and tcp[2:2] & 0xfc00!=0 and tcp[0:2]!=3389 and tcp[2:2]!=3389'
tcpdump -n -N -i eth1 'tcp[0:2]>1024 and tcp[2:2]>1024 and tcp[0:2]!=3389 and tcp[2:2]!=3389'
將tcpdump 的內容寫到檔案，以供tcpreplay 或 ethereal 來寫
tcpdump -n -i br0 ip host 170.116.11.94 and port 80 -w file
看某一個特定subnet
tcpdump -n -i br0 ip net 192.168.200.0/24
TCP Flags
URG ACK PSH RST SYN FIN
## tcp[13]==17 --> FIN, ACK
## tcp[13]==2 --> SYN
## tcp[13]==18 --> SYN, ACK
## tcp[13]==4 --> RST
# tcpdump -n -i br0 host 192.168.200.1 and tcp[13]==17 or tcp[13]=2
將tcpdump 的內容寫到檔案，以供tcpreplay 或 ethereal 來寫
tcpdump -n -i br0 ip host 170.116.11.94 and port 80 -w file
關於tcpdump 抓檔時, 所設定的-s snaplength 過小時, tcpreplay 在replay traffic 時, 會有的處理方式
tcpdump -s snaplen
tcpreplay -u
can be
pad -- pad the end of the packet with zeros
trunc -- re-adjusting the length in the IP header
-u or untruncate
When a packet is truncated in the capture file because the snaplen was too small, this option will pad the end of the packet with zeros, or truncate (trunc) it by re-adjusting the length in the IP header. The trunc option will only alter IPv4 packets, all others will be sent unmodified.
tcpreplay -R
tcpreplay -r
# tcpreplay -r 30
tcpreplay -m 0.1
Some early experience
//使用前須先執行，經過測試好像不用執行這行指令也可以work的樣子，不是很確定
//將eth0 interface設定為promisc mode，亦即網路上所有流經本地端的封包，都會被parsing
#ifconfig eth0 promisc
//將interface eth的promisc mode disable
#ifconfig eth0 -promisc
//不要使用promiscuous mode的情形下來節取packet，尚在測試中，不是很確定
#tcpdump -p
//dump source ip address 為 143.158.11.94的packet
#tcpdump src host 143.158.11.94
//dump source ip address & destination ipaddress為143.158.11.96的packet
#tcpdump host 143.158.11.96
//dump source port & destination port為80的packet
#tcpdump port 80
//dump destination network為143.158.11.0/24 的packet
# tcpdump dst net 143.158.11.0/24
# tcpdump -nvi eth2 ip net 172.16.0.0/16
>> dump a network wioth specified netmask
#./tcpdump -i wan1 -s 0 net 198.145.245.0 mask 255.255.255.0 -w aaa.pcap
>> or like this format
#./tcpdump -i wan1 -s 0 net 198.145.245.0/24 -w aaa.pcap
promiscuous mode 是指 ethernet card 接收全部 packet 的一種模式，正常的情況, ethernet card 是用 mode 3.
ethernet card 的receive mode 有:
01h turn off receiver
02h receive only packes sent to this interface
03h mode 2 plus broadcast packets
04h mode 3 plus limited multicast packets
05h mode 3 plus all multicast packets
06h all packets(promiscuous mode)
07h raw mode for serial line only(v1.10+)
你可能執行類似 tcpdump 的網路監聽程式, 才會有上述的 error mesg
網卡可以置一種模式叫混雜模式（promiscuous），在這種模式下工作的網卡能夠接收到一切通過它的數據，而不管實際上數 據的目的地址是不是他。這實際上就是我們SNIFF工作的基本原理：讓網卡接收一切他所能接收的數據。
Q. How to set network card in the promiscuous mode?
A.
# ifconfig eth0 promisc
// Be sure to replace "eth0" with your own network interface in case it's "wlan0" or something else.
// To remove promiscuous mode, type:
# ifconfig eth0 -promisc
ethereal/wireshark Analyzes => Display Filters can input the filtering rule to search the specified packets
e.g.
* tcp.flags.reset == 1
* tcp.len >= 1500
* frame.number==24333 (each packet is called a frame in the ethereal/wireshark, so we can use frame as the display filter component
* tcp.analysis.flags (use wireshark tcp analysis result)
* tcp.analysis.lost_segment
* tcp.analysis.retransmission
* tcp.analysis.fast_retransmission
* cdp.checksum_bad==1 || edp.checksum_bad==1 || ip.checksum_bad==1 || tcp.checksum_bad==1 || udp.checksum_bad==1
Advanced wireshark search tips
include:
Display filter:
e.g.
frame contains fe:3d:dd:36
or other display filtering rules
Hex value:
fe:3d:dd:36 or fe3ddd36
String:
"babala" (guessing usage...)
ipv6 filtering rules in the wireshark
icmpv6
ipv6.src
ipv6.dst
ipv6.addr == ff02::1:2
We can setup the coloring rule in the wireshark to separate the different packets type in the following method.
Select View -> Coloring Rules
Setup the coloring rules to display different color according the filtering rules.
wireshark will sniff all the packets destinated the Network Card buffer. And print the packets in the raw socket format.
We can disable the TCP checksum verification by the following steps:
1. Select Edit->Preferences
2. Select protocols -> TCP from the left frame of current window
3. Disable the option of "Validate the TCP checksum if possible"
Then the checksum error of TCP packets will be ignored by the wireshark.
TCP principle
* TCP often send an ACK packet while received a data packet, the acknowlegement number is the received packet sequence number plus received packet payload length.
* Sometimes TCP will send an Cumulative ACK if a preACK is not yet sent out because the data of sender is quickly enough (often less than 500 ms)
(According the RFC documents: TCP ACK generation[RFC 1122, RFC 2581])
* TCP will send a duplicated-ACK while it receive a miss-order packet (miss one packet and receive the afterward packet, wireshark call "TCP Previous segment lost").
* After receive a out-of-order packet (missing the previous packets, wireshark label "TCP Previous segment lost"), the wireshark will label all the following received packet "TCP Retransmission" until the packet sequence of previous received out-of-order packet.
* In wireshark, after receiving a packet labeled "
Dup ACK (Due to miss a packet of a next sequence number, this is labeled a "TCP Previous segment lost" in the wireshark)
Retransmission (Due to happen the specified RTT timeout and no ack received)
Fast Retransmission (Due to receive 3 ack packet with the same
Refer the Fast Retransmission description as following:
http://en.wikipedia.org/wiki/Fast_retransmit
Fast Retransmit is an enhancement to TCP which reduces the time a sender waits before retransmitting a lost segment.
A TCP sender uses timers to recognize lost segments. If an acknowledgement is not received for a particular segment within a specified time (a function of the estimated Round-trip delay time), the sender will assume the segment was lost in the network, and will retransmit the segment.
The fast retransmit enhancement works as follows: if a TCP sender receives three duplicate acknowledgements with the same acknowledge number (that is, a total of four acknowledgements with the same acknowledgement number), the sender can be reasonably confident that the segment with the next higher sequence number was dropped, and will not arrive out of order. The sender will then retransmit the packet that was presumed dropped before waiting for its timeout.
2008年10月7日 星期二
Linux Command - vi, use Vi to delete ^M signature
My experience
在Unix/BSD中，要顯示
^M等符號的方法，
按住Ctrl + V + M就會產生" ^M "的符號
實際例子：
當從windows傳送檔案到Unix/BSD之後，
或是Unix互傳檔案時，沒有使用ascii mode，而使用binary mode時
文字檔的每行行末會出現 ^M 的符號，這樣很難看，希望將這個符號刪除。
利用vi的替代功能
將^M改為空白符號
:%s/(Ctrl + V +M)//g
:1,$s/(Ctrl + V +M)//g
不過在剛拿到這些scripts(經由E-Mail得到)，發現其在DOM內用vi觀看時，會發現在每行的行尾都有一個^M符號，而在Linux Source下觀看則沒有，使得每次想run該script時，都必須先進vi，以人工的方式將這個符號刪除，但是由於資料是儲存在ramdisk中，故每次重新開機後，資料就會恢復原狀，這個問題困擾我滿久的。
因為剛開始就懷疑原因可能是UNIX和DOS在做文字檔轉換的時候，所多出來的符號，但是原本是朝向由vi內部去做符號搜尋替換的方向，或是由vi的特殊功能來改善，但是試過很多方法沒有效果。
不過現在發現這個工具”dos2unix”，在嘗試性的run過一遍後，竟然驚奇的發現原來的^M符號不見了，現在就將方法詳列於下
//指令格式dos2unix –n infile outfile
// If the dest file is new created file, you should add "-n" parameter
# dos2unix –n wanfo.html wanfo
// If the dest file is already existed, you should add "-o" parameter or just leave the parameter empty
# dos2unix -o wanfo.html
# dos2unix wanfo.html
原本wanfo.html內有^M符號，經過下面指令執行後，產生出來的wanfo，並沒有^M符號
some experiences of other advisor
http://newbiedoc.sourceforge.net/text_editing/vi.html#SEARCHING
在Unix/BSD中，要顯示
^M等符號的方法，
按住Ctrl + V + M就會產生" ^M "的符號
實際例子：
當從windows傳送檔案到Unix/BSD之後，
或是Unix互傳檔案時，沒有使用ascii mode，而使用binary mode時
文字檔的每行行末會出現 ^M 的符號，這樣很難看，希望將這個符號刪除。
利用vi的替代功能
將^M改為空白符號
:%s/(Ctrl + V +M)//g
:1,$s/(Ctrl + V +M)//g
不過在剛拿到這些scripts(經由E-Mail得到)，發現其在DOM內用vi觀看時，會發現在每行的行尾都有一個^M符號，而在Linux Source下觀看則沒有，使得每次想run該script時，都必須先進vi，以人工的方式將這個符號刪除，但是由於資料是儲存在ramdisk中，故每次重新開機後，資料就會恢復原狀，這個問題困擾我滿久的。
因為剛開始就懷疑原因可能是UNIX和DOS在做文字檔轉換的時候，所多出來的符號，但是原本是朝向由vi內部去做符號搜尋替換的方向，或是由vi的特殊功能來改善，但是試過很多方法沒有效果。
不過現在發現這個工具”dos2unix”，在嘗試性的run過一遍後，竟然驚奇的發現原來的^M符號不見了，現在就將方法詳列於下
//指令格式dos2unix –n infile outfile
// If the dest file is new created file, you should add "-n" parameter
# dos2unix –n wanfo.html wanfo
// If the dest file is already existed, you should add "-o" parameter or just leave the parameter empty
# dos2unix -o wanfo.html
# dos2unix wanfo.html
原本wanfo.html內有^M符號，經過下面指令執行後，產生出來的wanfo，並沒有^M符號
some experiences of other advisor
http://newbiedoc.sourceforge.net/text_editing/vi.html#SEARCHING
Searching and Replacing
:line1,line2s/old_string/new_string/g |
\r\n = chr(13)chr(10) = MS-DOS
Our MSDOS text file should look like this:
Friday the 13th^M |
And our mac text file should look like that
"Friday the 13th^M^M^MDearSir,^M^M...." |
MS-DOS/Windows -> UNIX conversion:
In order to remove these ugly ^M, you search for them and replace them by....nothing!
So first, let's search for those weird ^M ... but, how can you search for character 'ENTER'?
:1,$s/^V^M// |
(where ^V is Control-V, and ^M is ENTER or Control-M)
note that VI doesn't display the ^V, so you'll only see
:1,$s/^M// |
訂閱：
文章 (Atom)