测试程序：

// test.c
#include <stdio.h>
#include <pthread.h> 

int g_data = 0;

 void thread1_func(void *args)
 {
     int i = 0;
     while(i < 2)
     {
         i++;
         g_data += i;
     }
 }

 int main()
{
    int i;
    int flag = 1;

    printf("wait gdb\n");
    while(1 == flag);
    printf("jump wait gdb\n");


    for(i = 1; i <= 100; i ++)
    {
        g_data += i;
    }
    printf("g_data[1-100] = %d \n", g_data );

	pthread_t t1;
	pthread_t t2;
	pthread_create(&t1, NULL, (void*)thread1_func, NULL);
    pthread_join(t1, NULL);
    return 0;
}

编译：

// -g debug 模式
gcc test.c -g -lpthread -o test

1. 程序运行，根据 pid 打开 gdbserver
1.1 ps 查看程序pid进程
ps aux

jony       25651 98.2  0.0   2648   632 pts/0    R+   21:29   0:07 ./test
jony       25654  0.0  0.0  20324  3672 pts/1    R+   21:30   0:00 ps -aux

其中， 25651 即为我们想调试的程序pid。

1.2 打开gdbserver
sudo gdbserver --attach localhost:1234 25651
gdbserver：gdb服务器。 嵌入式环境下，需要交叉编译，一般放入/usr/bin目录。
--attach 表示调试正在运行的程序。
localhost:1234 表示gdbserver开放的接入ip和端口。这里也可以使用串口，改为形如/dev/ttyb，具体取决于linux /dev目录下串口名称。
25651 正在运行程序的pid。

至此，gdbserver 服务器已经打开，外部可以根据ip和端口进行远程调试。

2 远程调试
2.1 启动gdb
终端输入gdb，调试目标为嵌入式设备，使用交叉编译器自带的gdb，如arm-linux-gnueabihf-gdb,具体取决于所采用的交叉编译器。
2.2 连接远程gdbserver
(gdb) target remote localhost:1234
target remote：gdb连接远程命令
localhost:1234：远程ip、端口，也可以为串口。

输出结果：

Remote debugging using localhost:1234
Reading /mnt/hgfs/ubuntu20/test_code/gdb_test/test from remote target...
warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.
Reading /mnt/hgfs/ubuntu20/test_code/gdb_test/test from remote target...
Reading symbols from target:/mnt/hgfs/ubuntu20/test_code/gdb_test/test...
Reading /lib/x86_64-linux-gnu/libpthread.so.0 from remote target...
Reading /lib/x86_64-linux-gnu/libc.so.6 from remote target...
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
Reading symbols from target:/lib/x86_64-linux-gnu/libpthread.so.0...
Reading symbols from /usr/lib/debug/.build-id/e5/4761f7b554d0fcc1562959665d93dffbebdaf0.debug...
Reading symbols from target:/lib/x86_64-linux-gnu/libc.so.6...
Reading /lib/x86_64-linux-gnu/libc-2.31.so from remote target...
Reading /lib/x86_64-linux-gnu/.debug/libc-2.31.so from remote target...
Reading /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.31.so from remote target...
Reading /usr/lib/debug//lib/x86_64-linux-gnu/libc-2.31.so from remote target...
Reading symbols from target:/usr/lib/debug//lib/x86_64-linux-gnu/libc-2.31.so...
Reading symbols from target:/lib64/ld-linux-x86-64.so.2...
Reading /lib64/ld-2.31.so from remote target...
Reading /lib64/.debug/ld-2.31.so from remote target...
Reading /usr/lib/debug//lib64/ld-2.31.so from remote target...
Reading /usr/lib/debug/lib64//ld-2.31.so from remote target...
Reading target:/usr/lib/debug/lib64//ld-2.31.so from remote target...
(No debugging symbols found in target:/lib64/ld-linux-x86-64.so.2)
Reading /lib64/ld-linux-x86-64.so.2 from remote target...
main () at test.c:22
22	    while(1 == flag);

可以发现程序停留在 22行死循环。我们可以通过set指令改变变量状态，使程序继续运行调试。

(gdb) set flag=0
(gdb) n
23	    printf("jump wait gdb\n");

至此，可以通过gdb命令远程调试程序了。

3. 简单调试
3.1 l 查看查看代码提示

(gdb) l 
17	{
18	    int i;
19	    int flag = 1;
20	
21	    printf("wait gdb\n");
22	    while(1 == flag);
23	    printf("jump wait gdb\n");
24	
25	
26	    for(i = 1; i <= 100; i ++)

3.2 b 程序断点

(gdb) b 35
Breakpoint 1 at 0x5574c45e7297: file test.c, line 35.

3.3 watch 放置一个观察点，当变量被读出或写入时程序被暂停

(gdb) watch g_data 
Hardware watchpoint 2: g_data

c 程序继续运行， g_data 发生改变，程序暂停。

(gdb) c
Continuing.

Hardware watchpoint 2: g_data

Old value = 105
New value = 120

Hardware watchpoint 3: g_data

Old value = 105
New value = 120
main () at test.c:26
26	    for(i = 1; i <= 100; i ++)

3.3 whatis 判断变量类型

(gdb) whatis i
type = int

3.4 查看断点、观测点，删除观测点

(gdb) info breakpoints 
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00005574c45e7297 in main at test.c:35
2       hw watchpoint  keep y                      g_data
	breakpoint already hit 12 times
3       hw watchpoint  keep y                      g_data
	breakpoint already hit 12 times
(gdb) delete breakpoints 2
(gdb) delete breakpoints 3
(gdb) info breakpoints 
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x00005574c45e7297 in main at test.c:35

3.5 进入线程

(gdb) b 11 //线程内打断点
Breakpoint 1 at 0x55d837bd11de: file test.c, line 11.
(gdb) info breakpoints  // 查看断点
Num     Type           Disp Enb Address            What
1       breakpoint     keep y   0x000055d837bd11de in thread1_func at test.c:11
2       breakpoint     keep y   0x000055d837bd1297 in main at test.c:35
gdb) c
Continuing.
[New Thread 25911.25926]

Thread 1 "test" hit Breakpoint 2, main () at test.c:35
35	    pthread_join(t1, NULL);

(gdb) info threads	// 查看线程信息
  Id   Target Id                 Frame 
* 1    Thread 25911.25911 "test" main () at test.c:35
  2    Thread 25911.25926 "test" thread1_func (args=0x0) at test.c:11
(gdb) thread 2  // 进入线程 2
[Switching to thread 2 (Thread 25911.25926)]
#0  thread1_func (args=0x0) at test.c:11
11	         i++;

//  线程内调试，查看数据
gdb) p i
$2 = 0
(gdb) p g_data 
$3 = 5050
(gdb) n

Thread 2 "test" hit Breakpoint 1, thread1_func (args=0x0) at test.c:11
11	         i++;
(gdb) n
12	         g_data += i;
(gdb) n
9	     while(i < 2)
(gdb) p g_data 
$4 = 5051

3.6 捕捉信号

(gdb) catch signal 
Display all 149 possibilities? (y or n)
(gdb) catch signal SIGINT
Catchpoint 1 (signal SIGINT)
(gdb) info breakpoints 
Num     Type           Disp Enb Address            What
1       catchpoint     keep y                      signal "SIGINT" 
(gdb) n
^C
Catchpoint 1 (signal SIGINT), 0x000056160ced322f in main () at test.c:22
22	    while(1 == flag);

3.7 until + 行数， 运行到某行

(gdb) u 35
main () at test.c:35
35	    pthread_join(t1, NULL);
(gdb) p g_data 
$1 = 5053

3.8 多进程调试
代码fork之后，gdb默认进入parent进程，通过如下代码查看fork跟踪进程方式。

(gdb) show follow-fork-mode
Debugger response to a program call of fork or vfork is "parent".

修改为fork跟踪子进程：

(gdb) set follow-fork-mode child
(gdb) show follow-fork-mode
Debugger response to a program call of fork or vfork is "child".

另外，子进程也可通过 gdb attach pid进入。