xuboso's Blog

goroutine

2024-12-05

goroutine

how to limit goroutines use semaphore

https://bostonc.dev/blog/go-semaphore

https://gist.github.com/jboursiquot/ef53edb815fe18c8160dbe8504dd9d60

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
var (
  concurrency = 5
  semaChan = make(chan struct{}, concurrency)
)

func doWork(item int) {
  semaChan <- struct{}{} // block while full
  go func() {
    defer func() {
      <-semaChan // read releases a slot
    }()
    // work
    log.Println(item)
    time.Sleep(1 * time.Second)
  }()
}

func main(){
  for i := 0; i <= 1000; i++ {
    doWork(i)
  }
}

https://medium.com/deep-golang/8-python-performance-tips-i-discovered-after-years-of-coding-in-golang-764375658d90

https://medium.com/deep-golang/5-go-concurrency-patterns-i-wish-id-picked-up-sooner-b1b7dae6d71e

https://medium.com/@ahamrouni/mastering-concurrency-in-go-from-goroutines-to-semaphores-123fdd150213

参考

著名的 one million websockets

https://github.com/eranyanay/1m-go-websockets

https://www.bomberbot.com/golang/scaling-to-a-million-websockets-with-go/

https://awesome-go.com/goroutines/

https://ofeng.org/posts/goroutine-pool/

https://go.libhunt.com/categories/493-goroutines

https://medium.com/deep-golang

awesome projects

2024-12-04

一些有趣的项目

mkcert

mkcert 可以方便的为你本地开的发域名配置 SSL
https://github.com/FiloSottile/mkcert

Preact js

Preact 在 DOM 之上提供了最薄的虚拟 DOM 抽象，在提供稳定的平台特性和注册事件处理程序的同时还确保与其他库无缝兼容。

Preact 无需转译即可在浏览器中直接使用。

https://preactjs.com/

developer-roadmap

developer-roadmap 包含了成为一名开发者的学习路线图，涵盖前端和后端的各种语言
里面紫色标识的是优先或者必须考虑学习的。

https://github.com/kamranahmedse/developer-roadmap

https://roadmap.sh/backend

astro

astro 可以生成内容驱动的网站，是建立 blog 的好选择
The web framework for content-driven websites
https://astro.build/

mermaid

mermaid.js 可以让你写代码一样生成流程图，时序图，类图，ER 图等。
只需要简单的描述，可以结合 AI 大模型为你生成 mermaid 语法。
https://mermaid.js.org/

mermaid playground

Deno

Deno 是一个 javascript 运行时，类似于 V8。可以让你轻松的书写现代 JavaScript 语法。
自带支持 TypeScript

https://deno.com/

awesome github

汇集了 github 出色的项目
https://github.com/sindresorhus/awesome

design-resources

design resources for developers 汇集了开发者用到的设计资源
https://github.com/bradtraversy/design-resources-for-developers

public apis

一些公共的 API
https://github.com/public-apis/public-apis

hacking resources

hacker 资源
https://github.com/vitalysim/Awesome-Hacking-Resources

instrctales

Instructables 是一个专门提供用户创建和上传的 DIY 项目教程的网站

https://www.instructables.com/20-Awesome-Projects/

realword

https://github.com/gothinkster/realworld

系统架构设计

https://github.com/donnemartin/system-design-primer

fastify

fastify 是基于 node.js 的高效 web 框架
https://github.com/fastify/fastify
https://fastify.dev/

supervision

计算机可视化
https://github.com/roboflow/supervision

github 上的后端开发资源

https://github.com/topics/backend

Temporal Workflow

Temporal 是一个新兴的分布式的工作流引擎，可用于解决跨服务、跨时间周期的复杂业务流程等。
https://temporal.io/
https://code2life.top/blog/0070-temporal-notes

Terraform

Terraform 是一款由 HashiCorp 公司开发的开源基础设施即代码（Infrastructure as Code，简称 IaC）工具。
包含基础设施自动化部署， 云服务资源管理，基础设施状态管理，版本控制集成，多环境一致性保障

https://www.terraform.io/

neofetch

neofetch 是一个漂亮的打印系统信息的命令行工具

https://github.com/dylanaraps/neofetch

这人的 Shell 玩的飞起！！！
https://github.com/dylanaraps

ligo 离线翻译

https://github.com/thewh1teagle/lingo

离线翻译大模型

https://huggingface.co/docs/transformers/main/en/model_doc/m2m_100#overview

引用

https://dev.to/olucasandrade/10-awesome-github-repositories-that-everyone-should-know-2pcp

https://awesome-cp-projects.github.io/

https://github.com/zhashkevych/awesome-backend

https://github.com/shahednasser/awesome-resources

https://github.com/fastify/fastify

https://getpino.io/#/

https://roadmap.sh/backend/project-ideas

https://dev.to/durgesh4993/10-github-repos-to-improve-as-a-backend-engineer-4lmi

https://www.boot.dev/tracks/backend

https://blog.boot.dev/backend/best-backend-projects/

低代码开发平台

少数派

https://github.com/Tencent/APIJSON

https://github.com/ashishps1/awesome-system-design-resources

https://github.com/binhnguyennus/awesome-scalability

https://github.com/dylanaraps/pure-bash-bible

go内存对齐

2024-11-25

我写什么呢，前辈们都写的那么好

参考

https://geektutu.com/post/hpg-struct-alignment.html

https://eddycjy.gitbook.io/golang/di-1-ke-za-tan/go-memory-align

https://golang.dbwu.tech/performance/memory_alignment/

https://ueokande.github.io/go-slice-tricks/

https://go.dev/wiki/

https://go.dev/wiki/All

https://tehub.com/a/c2XXL0xk0Q

https://yunsonbai.top/2023/03/07/go-memalign-struct/

https://gfw.go101.org/article/memory-layout.html

https://blog.haohtml.com/archives/19040

https://blog.alovn.cn/2021/02/02/golang-memory-alignment/

linux常用命令

2024-11-22 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
#显示系统信息
uname -a
cat /proc/cpuinfo  

#显示磁盘空间使用情况
df -h

#显示当前目录磁盘占用情况
du -sh

#按照大小列出当前文件夹前10个磁盘占用
du -h / | sort -hr | head -n 10

#内存占用情况排序
ps aux --sort -rss
ps aux --sort -%mem
ps aux | awk '{print $6/1024 " MB\t\t" $11}' | sort -rn

#从远程服务器拷贝文件到本地目录
scp user@ip:/remote_path /local_path

#将文件拷贝到远程服务器
scp file.txt user@ip:/remote_path

#在目录下查找含有某个字符串的文件
find /folder_name -name 'prefix*'

grep -rnw '/path/to/somewhere/' -e 'pattern'

grep -Ril "text-to-find-here" /

grep -r "string to be searched" /path/to/dir

#查找目录下大于多少M的文件
find /folder_name -size +100M

#为当前目录创建压缩包
tar czf archive.tar.gz directory

#解压文件
tar xzf archive.tar.gz

#创建一个软连接
ln -s source_file link

#列出用户打开的文件
lsof -u user

#列出端口占用情况
lsof -i:80
lsof -i -P -n | grep LISTEN
netstat -tulpn | grep LISTEN
nc -vz 192.168.2.254 80

#检查域名或IP服务器端口打开情况
nmap baidu.com
nmap 13.384.239.67

#awk命令
#-F设置分隔符是: 默认是,
cat /etc/passwd | awk -F: '{print $1,$6}'

#OFS设置输出的连接符
date | awk 'OFS="-" {print$6,$2,$3}'

#输出第一列
who | awk '{print $1}'

#git切换到上一个分支
git checkout -

#docker查看容器内存占用情况
docker stats --no-stream

#docker显示磁盘使用情况
docker system df -v


终端录制

Terminal session recorder

crontab调试工具

https://crontab.guru/

参考引用

https://www.cyberciti.biz/faq/unix-linux-check-if-port-is-in-use-command/

Shell Tutorial

https://www.tecmint.com/free-online-linux-learning-guide-for-beginners/

linux awk

https://www.linuxtrainingacademy.com/linux-commands-cheat-sheet/

https://kubernetes.io/zh-cn/

常用命令

https://github.com/mingongge/BestSRE

计算机网络

go string and []byte

2024-11-22

先看[]byte的数据结构

1
2
3
4
5
type slice struct {
  data uintptr
  len int
  cap int
}

string的数据结构

1
2
3
4
type string struct {
  data uintptr
  len int
}

String不可变，[]byte可变

String默认是UTF-8编码, []byte没有

由于String类型是不可变的，如果经常需要拼接字符等，可以使用strings.Builder方法

[]byte适合二进制传输，尤其适合文件，网络传输，因为网络就是按字节传输的

[]byte转换string

1
2
3
4
5
6
data := []byte{72, 101, 108, 108, 111, 44, 32, 87, 111, 114, 108, 100, 33}
text := string(data)

data := "Hello, World!"
b := []byte(data)     // string转换[]byte
s := string(b)        // []byte转换string

[]rune是int32类型，默认占四个字节，即使是ASCII字符一样，所以比string和[]byte占用的内存空间更大。

Use strings for simplicity and Unicode support,
bytes for mutability and versatility,
and runes for character-level operations.

参考

https://go.dev/blog/strings

https://en.wikipedia.org/wiki/UTF-8

https://medium.com/@tyler_brewer2/bits-bytes-and-byte-slices-in-go-8a99012dcc8f

https://www.joelonsoftware.com/2003/10/08/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character-sets-no-excuses/

https://www.codingexplorations.com/blog/understanding-strings-bytes-and-runes-in-go-advantages-and-disadvantages

https://100go.co/20-slice/

https://syslog.ravelin.com/byte-vs-string-in-go-d645b67ca7ff

https://www.ekwbtblog.com/entry/2023/03/20/080011