Reading Source Code: Bilibili in Go

subtitle: “Learning design pattern in large-scale code”

Before Start

Writing machine code is sometimes far more easier than writing in human language because there are something more than just the facts to record for human’s comprehension. This is probabaly one of the reasons that all the documentation sucks in software engineering and it’s much better to read the source code than sticking to one piece of documentation that may be outdated. IMO the ability to read source code is one of the skills required to become a good software engineer.

There are already several posts asking about what to read to get better in Go like this Reddit post but I think here are some general approaches that can be applied to reading source code in all languages. This blog is going to implement such methods to read said to be leaked Bilibili source code.

1. Know Design Patterns. Common design patterns adopted in other languages are usually useful in Golang.
2. Some Standard Libs. These codes are easily done and relatively easy to read compared to comprehensive project codes. And sometimes it’s easier to understand what the project code is really about if you know the underlying protocols in the libraries. Just go to golang/pkg and simply start somewhere.
3. Just start reading. Sometimes the most important thing is to get started. There are a set of resources online like tour.golang and effective go that can help you get started.
4. Practice. The best way to learn sth to do it yourself. So just set up the language env locally and try writing simple snippets of code.

Reading…

We will start with some fun parts and move gradually to the important designs.

Product Manager VS Developer

There’s always an on-going war between product managers and software developers, especially when a new feature required. I simply searched fuck in the project repo and here’s all related programs. It’s clearly to see all the keywords related LOL.

DAO

Data service generally follows readable, scalable and maintainable patterns. The entity - dao - service layer model is a generally good way for definition.

Intresting thing is bilibili uses MySQL instead of PostgreSQL, which is recommended by the Golang. An easy way to install is directly using MySQL mirror in Docker:

$ sudo docker pull mysql # : can also add version tag
$ sudo docker images
$ sudo docker run -p 3306:3306 --name mysql2 -e MYSQL_ROOT_PASSWORD=root -d mysql
$ sudo docker ps

Persistence & DB Access

One common way to do such is dependency injection by explicitly passing a connection pool pointer to our HTTP handlers and then onward to database logic. So defining handeler that have access to application-level items are crucial.

DB Master

func newDBMasterInfo(addr string, c *conf.MasterInfoConfig) (*dbMasterInfo, error) {
	m := &dbMasterInfo{c: c, addr: addr}
	conn, err := client.Connect(c.Addr, c.User, c.Password, c.DBName)
	if err != nil {
		log.Error("db master info client error(%v)", err)
		return nil, errors.Trace(err)
	}
	defer conn.Close()
	if m.c.Timeout > 0 {
		conn.SetDeadline(time.Now().Add(m.c.Timeout * time.Second))
	}
	r, err := conn.Execute("SELECT addr,bin_name,bin_pos FROM master_info WHERE addr=?", addr)
	if err != nil {
		log.Error("new db load master.info error(%v)", err)
		return nil, errors.Trace(err)
	}
	if r.RowNumber() == 0 {
		if m.c.Timeout > 0 {
			conn.SetDeadline(time.Now().Add(m.c.Timeout * time.Second))
		}
		if _, err = conn.Execute("INSERT INTO master_info (addr,bin_name,bin_pos) VALUE (?,'',0)", addr); err != nil {
			log.Error("insert master.info error(%v)", err)
			return nil, errors.Trace(err)
		}
	} else {
		m.addr, _ = r.GetStringByName(0, "addr")
		m.binName, _ = r.GetStringByName(0, "bin_name")
		bpos, _ := r.GetIntByName(0, "bin_pos")
		m.binPos = uint32(bpos)
	}
	return m, nil
}

Lottery Result

Pretty interesting yet common way to hype the room lol.

defer func() {
		//resp不为nil 说明抽奖成功 ，发送弹幕
		if m != nil {

			if resp.GetJoin() != nil {
				if err1 := s.task.Do(metadata.WithContext(ctx), func(ctx context.Context) {
					s.sendDamu(ctx, req.GetRoomid(), req.GetMid(), m["content"], req.GetPlatform())
				}); err1 == fanout.ErrFull {
					log.Info("sendDamu_task_is_full roomid= %d ", req.GetRoomid())
					s.sendDamu(ctx, req.GetRoomid(), req.GetMid(), m["content"], req.GetPlatform())
				}
			} else {
				//抽奖不成功也要发送弹幕，概率20% 造成一种很多人中奖的假象
				rand.Seed(time.Now().Unix())
				if s.canSendDamu(ctx, req.GetId(), req.GetMid()) && rand.Intn(5) == 2 {
					if err1 := s.task.Do(metadata.WithContext(ctx), func(ctx context.Context) {
						s.sendDamu(ctx, req.GetRoomid(), req.GetMid(), m["content"], req.GetPlatform())
					}); err1 == fanout.ErrFull {
						log.Info("sendDamu_task_is_full roomid= %d ", req.GetRoomid())
						s.sendDamu(ctx, req.GetRoomid(), req.GetMid(), m["content"], req.GetPlatform())
					}
				}
			}
		}

	}()

Blah Blah Blah

B站, 一个致力于推动狗语言应用的视频交友网站, 今年4月22号的瓜让大家都吃得很开心. 作为一个从中学起就是忠实用户并立志成为码农的渣渣, 认为十分有必要拜读下集广大B站开发前辈智慧与膜法功力的源码.