viper作為配置框架,其功能非常的強大,我們沒有理由不去了解一下。我們先看官網對它的功能簡介:
viper是完整配置解決方案,他可以處理所有類型和格式的配置文件,他有如下功能:
- 設置默認配置
- 支持讀取 JSON TOML YAML HCL 和 Java 屬性配置文件
- 監聽配置文件變化,實時讀取讀取配置文件內容
- 讀取環境變量值
- 讀取遠程配置系統 (etcd Consul) 和監控配置變化
- 讀取命令 Flag 值
- 讀取 buffer 值
- 讀取確切值
乍一看,未免有相見恨晚之感,可仔細一想,不免腦袋里有另外一種聲音:不會不支持讀取 zookeeper
吧?好吧,至少我是這樣的。
基于這種想法,當然要去立馬嘗試,如下:
1
|
viper.AddRemoteProvider("zookeeper", "xx.xx.xx.xx:2181", "/viper/test") |
返回結果是:
Unsupported Remote Provider Type zookeeper
果不其然,于是追蹤 viper.AddRemoteProvider
的源碼,發現viper只支持如下幾種
1
|
var SupportedRemoteProviders = []string{"etcd", "consul", "firestore"} |
如果就此打住,未免有點太可惜,作為偏執狂,總想著能否來改造下viper,讓其支持 zookeeper
,于是在issue上找是否有人遇到同樣的問題,還整讓我找到了, 傳送 。但是不完整,且稍微有點bug。所以根據他的基礎上,我做了些調整。進入正題,我們開始修改viper源碼。說明下,我的viper版本是最新的 1.7.0
修改源碼
1、添加zookeeper.go
添加的位置: github.com/bketelsen/crypt/zookeeper
, zookeeper
目錄需要自己創建, github.com/bketelsen/crypt
是viper的依賴包,會自動下載
文件內容:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
|
package zookeeper import ( "errors" "fmt" zk "github.com/samuel/go-zookeeper/zk" //"github.com/xordataexchange/crypt/backend" "github.com/bketelsen/crypt/backend" "strings" "time" ) type Client struct { client *zk.Conn waitIndex uint64 } func New(machines []string) (*Client, error) { zkclient, _, err := zk.Connect(machines, time.Second) if err != nil { return nil, err } return &Client{zkclient, 0}, nil } func (c *Client) Get(key string) ([]byte, error) { resp, _, err := c.client.Get(key) if err != nil { return nil, err } return []byte(resp), nil } func nodeWalk(prefix string, c *Client, vars map[string]string) error { l, stat, err := c.client.Children(prefix) if err != nil { return err } if stat.NumChildren == 0 { b, _, err := c.client.Get(prefix) if err != nil { return err } vars[prefix] = string(b) } else { for _, key := range l { s := prefix + "/" + key _, stat, err := c.client.Exists(s) if err != nil { return err } if stat.NumChildren == 0 { b, _, err := c.client.Get(s) if err != nil { return err } vars[s] = string(b) } else { nodeWalk(s, c, vars) } } } return nil } func (c *Client) GetValues(key string, keys []string) (map[string]string, error) { vars := make(map[string]string) for _, v := range keys { v = fmt.Sprintf("%s/%s", key, v) v = strings.Replace(v, "/*", "", -1) _, _, err := c.client.Exists(v) if err != nil { return vars, err } if v == "/" { v = "" } err = nodeWalk(v, c, vars) if err != nil { return vars, err } } return vars, nil } func (c *Client) List(key string) (backend.KVPairs, error) { var list backend.KVPairs resp, stat, err := c.client.Children(key) if err != nil { return nil, err } if stat.NumChildren == 0 { return list, nil } entries, err := c.GetValues(key, resp) if err != nil { return nil, err } for k, v := range entries { list = append(list, &backend.KVPair{Key: k, Value: []byte(v)}) } return list, nil } func (c *Client) createParents(key string) error { flags := int32(0) acl := zk.WorldACL(zk.PermAll) if key[0] != '/' { return errors.New("Invalid path") } payload := []byte("") pathString := "" pathNodes := strings.Split(key, "/") for i := 1; i < len(pathNodes); i++ { pathString += "/" + pathNodes[i] _, err := c.client.Create(pathString, payload, flags, acl) // not being able to create the node because it exists or not having // sufficient rights is not an issue. It is ok for the node to already // exist and/or us to only have read rights if err != nil && err != zk.ErrNodeExists && err != zk.ErrNoAuth { return err } } return nil } func (c *Client) Set(key string, value []byte) error { err := c.createParents(key) if err != nil { return err } _, err = c.client.Set(key, []byte(value), -1) return err } func (c *Client) Watch(key string, stop chan bool) <-chan *backend.Response { respChan := make(chan *backend.Response, 0) go func() { for { resp, _, watch, err := c.client.GetW(key) if err != nil { respChan <- &backend.Response{nil, err} time.Sleep(time.Second * 5) } select { case e := <-watch: if e.Type == zk.EventNodeDataChanged { resp, _, err = c.client.Get(key) if err != nil { respChan <- &backend.Response{nil, err} } c.waitIndex = 0 respChan <- &backend.Response{[]byte(resp), nil} } } } }() return respChan } |
這個文件是實現 ConfigManager
接口,我們在上圖中看到 etcd
, consul
, filestore
,均有實現該接口,接口的定義很簡單
1
2
3
4
5
6
|
type ConfigManager interface { Get(key string) ([]byte, error) List(key string) (KVPairs, error) Set(key string, value []byte) error Watch(key string, stop chan bool) <-chan *Response } |
2、修改config.go
文件的位置: github.com/bketelsen/crypt/config/config.go
,如下圖
在 func NewStandardEtcdConfigManager(machines []string) (ConfigManager, error)
方法下面添加如下方法:
1
2
3
4
5
6
7
8
9
|
// NewStandardZookeeperConfigManager returns a new ConfigManager backed by Zookeeper. // Data will be encrypted. func NewStandardZookeeperConfigManager(machines []string) (ConfigManager, error) { store, err := zookeeper.New(machines) if err != nil { return nil, err } return NewStandardConfigManager(store) } |
在 func NewEtcdConfigManager(machines []string, keystore io.Reader) (ConfigManager, error)
方法下面添加如下方法:
1
2
3
4
5
6
7
8
9
|
// NewZookeeperConfigManager returns a new ConfigManager backed by zookeeper. // Data will be encrypted. func NewZookeeperConfigManager(machines []string, keystore io.Reader) (ConfigManager, error) { store, err := zookeeper.New(machines) if err != nil { return nil, err } return NewConfigManager(store, keystore) } |
這兩個方法是初始化 ConfigManager
對象,也就是我們剛才添加的 zookeeper.go
文件的對象
3、修改remote.go
文件的位置: github.com/spf13/viper/remote/remote.go
,如下圖
找到74行,用下面的代碼替換 func getConfigManager(rp viper.RemoteProvider) (crypt.ConfigManager, error)
方法
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
|
func getConfigManager(rp viper.RemoteProvider) (crypt.ConfigManager, error) { var cm crypt.ConfigManager var err error if rp.SecretKeyring() != "" { var kr *os.File kr, err = os.Open(rp.SecretKeyring()) if err != nil { return nil, err } defer kr.Close() switch rp.Provider() { case "etcd": cm, err = crypt.NewEtcdConfigManager([]string{rp.Endpoint()}, kr) case "zookeeper": cm, err = crypt.NewZookeeperConfigManager([]string{rp.Endpoint()}, kr) case "firestore": cm, err = crypt.NewFirestoreConfigManager([]string{rp.Endpoint()}, kr) default: cm, err = crypt.NewConsulConfigManager([]string{rp.Endpoint()}, kr) } } else { switch rp.Provider() { case "etcd": cm, err = crypt.NewStandardEtcdConfigManager([]string{rp.Endpoint()}) case "zookeeper": cm, err = crypt.NewStandardZookeeperConfigManager([]string{rp.Endpoint()}) case "firestore": cm, err = crypt.NewStandardFirestoreConfigManager([]string{rp.Endpoint()}) default: cm, err = crypt.NewStandardConsulConfigManager([]string{rp.Endpoint()}) } } if err != nil { return nil, err } return cm, nil } |
細心的讀者可能已經發現,其實就添加了兩個case選項:
4、修改viper.go
文件的位置: github.com/spf13/viper/viper.go
,如下圖
取+監聽zookeeper(1)\image-20200521222843002.png)
找到兩個 SupportedRemoteProviders
定義的定法,1.7.0版本的行號分別是:290,331。只要添加 zookeeper
,即可
1
|
SupportedRemoteProviders = []string{"etcd", "consul", "firestore", "zookeeper"} |
好了,修改代碼的工作已經完了,接下來我們來測試:
測試
注意:zookeeper中已經設置了內容
set /viper/test {"appName":"test","nodes":["127.0.0.1","127.0.0.2","127.0.0.3"]}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
|
package main import ( "fmt" "github.com/fsnotify/fsnotify" "github.com/spf13/viper" _ "github.com/spf13/viper/remote" "time" ) type config struct { AppName string Nodes []string } func main() { var waitGroup=sync.WaitGroup{} waitGroup.Add(1) readRemoteZookeeper() go watchRemoteZookeeper() waitGroup.Wait() } func readRemoteZookeeper() { viper.AddRemoteProvider("zookeeper", "62.234.15.24:2181", "/viper/test") viper.SetConfigType("json") err := viper.ReadRemoteConfig() if err != nil { panic(fmt.Sprintf("read remote zookeeper error:+%v", err)) } var C config viper.Unmarshal(&C) fmt.Printf("從zookeeper讀取配置內容:%+v\n", C) } func watchRemoteZookeeper() { go func() { for { //delay after each request time.Sleep(time.Second * 5) err := viper.WatchRemoteConfig() if err != nil { fmt.Errorf("unable to read remote config: %v", err) continue } fmt.Printf("從zookeeper讀取更新內容:appName=%s,nodes=%+v\n", viper.Get("appName"), viper.Get("nodes")) } }() } |
輸出內容:
從zookeeper讀取配置內容:{AppName:test Nodes:[127.0.0.1 127.0.0.2 127.0.0.3]}
從zookeeper讀取更新內容:appName=test,nodes=[127.0.0.1 127.0.0.2 127.0.0.3]
如果我們修改zookeeper的內容,則viper會讀取到更新后的內容:
1
|
set /viper/test {"appName":"test","nodes":["127.0.0.1","127.0.0.2","127.0.0.3","127.0.0.4"]} |
1
|
從zookeeper讀取更新內容:appName=test,nodes=[127.0.0.1 127.0.0.2 127.0.0.3 127.0.0.4] |
結語
讓viper支持 zookeeper
并不復雜的,并且基本上不需要修改原有的方法, 這要歸結于viper用到一個非常重要的設計原則: 開閉原則 ,讀者可以自行體會。
關于viper的基本使用, github 已經有非常詳細的例子,這里就不再贅述,如有疑問,可以私信我
到此這篇關于viper配置框架的介紹支持zookeeper的讀取和監聽的文章就介紹到這了,更多相關viper配置框架支持zookeeper的讀取和監聽內容請搜索服務器之家以前的文章或繼續瀏覽下面的相關文章希望大家以后多多支持服務器之家!
原文鏈接:https://studygolang.com/articles/28829