Skip to content

Instantly share code, notes, and snippets.

@yuuki
Created March 17, 2023 15:37
Show Gist options
  • Select an option

  • Save yuuki/4cc6524951f36a5ebbfe3daba728c5e4 to your computer and use it in GitHub Desktop.

Select an option

Save yuuki/4cc6524951f36a5ebbfe3daba728c5e4 to your computer and use it in GitHub Desktop.
ChatGPT prompt for Fault Localization in Microservices (Sock Shop)
あなたはマイクロサービスアプリケーションSock ShopのSREです。
Sock Shopはcarts,catalogue,front-end,orders,user,payment,shippingの7個のマイクロサービスにより構成されています。
以下は、Sock Shopのネットワーク通信の依存関係をmarmaid記法で表現したものです。
flowchart LR
front[front-end]
front --> orders[orders]
subgraph Orders
orders --> ordersdb[orders-db]
end
front --> carts[carts]
subgraph Carts
carts --> cartsdb[carts-db]
end
front --> user[user]
subgraph User
user[user] --> userdb[user-db]
end
front --> cata[catalogue]
subgraph Catalogue
cata --> catadb[catalogue-db]
end
orders --> carts
orders --> user
orders --> pay[payment]
orders --> ship[shipping]
以下のcsvは、front-endサービスのlatencyが上昇したときの各マイクロサービスのlatency、throughput、errorsの各メトリクスの時系列データを示しています。
上述のネットワーク依存関係を基に、これらの時系列データを解析し、どのメトリクスが原因であるかを調査してください。
,s-carts_throughput,s-carts_latency,s-catalogue_throughput,s-catalogue_latency,s-front-end_throughput,s-front-end_errors,s-front-end_latency,s-orders_throughput,s-orders_latency,s-payment_throughput,s-payment_latency,s-shipping_throughput,s-shipping_latency,s-user_throughput,s-user_latency
0,138.6,0.0052,69.9333,0.0045,307.7111,4.8,0.0335,31.5556,0.0224,35.2667,0.0003,31.2667,0.0011,242.9778,0.004
1,138.6444,0.0053,70.1111,0.0045,307.9111,4.7111,0.0335,31.2,0.0216,35.3556,0.0003,31.2667,0.0011,243.0889,0.004
2,137.9556,0.0053,69.8667,0.0044,307.5333,4.8444,0.0337,31.4667,0.0222,35.3333,0.0004,30.9333,0.0011,242.9778,0.0041
3,137.9111,0.0054,69.6667,0.0044,306.3556,4.0889,0.0344,31.2889,0.0218,35.2,0.0004,31.6444,0.0011,242.2444,0.0041
4,137.2889,0.0055,69.5556,0.0045,304.7556,5.5111,0.0347,30.3556,0.0225,35.1333,0.0004,30.7111,0.0011,241.6222,0.0043
5,138.0,0.0056,69.6222,0.0045,304.4,5.8444,0.0347,30.2,0.0222,35.1333,0.0004,30.2444,0.0012,241.5111,0.0042
6,137.3333,0.0059,69.4889,0.0045,304.0667,6.4667,0.0346,29.8444,0.0233,35.1556,0.0004,29.8222,0.0012,241.4889,0.0041
7,138.2,0.0061,69.8667,0.0046,306.2444,5.3333,0.0343,30.8444,0.0237,35.2222,0.0003,30.7111,0.0012,242.4444,0.004
8,137.0889,0.0062,69.0667,0.0047,305.3111,5.7111,0.0345,30.0444,0.0253,35.2667,0.0004,30.4,0.0012,242.0444,0.0042
9,137.1556,0.0057,69.6,0.0048,304.9333,5.2889,0.0349,30.8222,0.0251,35.2,0.0004,30.8889,0.0013,241.5556,0.0042
10,134.0,0.0076,67.7556,0.005,298.5556,5.3111,0.0374,30.0889,0.0271,34.5333,0.0004,30.2444,0.0012,235.7333,0.0043
11,135.2222,0.0075,68.8,0.005,298.8889,5.2,0.0374,30.6889,0.0252,34.4222,0.0004,30.2667,0.0011,237.2222,0.0041
12,134.1111,0.0076,67.7778,0.0049,298.5778,5.3333,0.0378,29.7556,0.0274,34.4667,0.0004,30.0444,0.0014,236.4667,0.0042
13,137.0,0.0054,69.2889,0.0046,303.0,5.4667,0.0356,30.3556,0.0253,34.8222,0.0004,29.9778,0.0015,240.7111,0.0041
14,135.4444,0.0055,68.4,0.0046,301.5111,4.6222,0.0365,30.5111,0.0251,34.6889,0.0004,30.8,0.0014,237.9556,0.0041
15,137.3556,0.0054,69.4444,0.0046,303.8222,4.3111,0.0355,31.3333,0.0235,34.8444,0.0004,31.0444,0.0011,240.3111,0.0041
16,137.0444,0.0055,69.3111,0.0046,305.5556,4.6,0.0347,30.9778,0.0244,35.0667,0.0004,31.5111,0.0013,241.5333,0.0041
17,138.5111,0.0053,70.2444,0.0046,307.0222,5.4222,0.0339,31.1111,0.0245,35.3333,0.0004,30.7111,0.0013,242.7333,0.0043
18,137.9778,0.0053,69.9556,0.0046,306.0889,6.3111,0.0339,30.0889,0.0261,35.3556,0.0004,30.4222,0.0016,242.9778,0.0042
19,137.8667,0.0054,69.8667,0.0045,305.2444,6.0889,0.0343,30.4,0.0257,35.2222,0.0004,30.0889,0.0015,241.9111,0.0043
20,137.4667,0.0054,69.8,0.0045,306.0667,5.6889,0.0341,30.4444,0.0259,35.3333,0.0004,30.8444,0.0015,242.7556,0.0042
21,137.3556,0.0055,69.3778,0.0045,305.1333,5.3778,0.0344,30.9111,0.0247,35.1556,0.0004,30.4444,0.0013,241.5111,0.0043
22,137.2222,0.0056,69.6222,0.0047,304.6222,5.2889,0.035,30.9556,0.0241,35.1111,0.0004,30.9556,0.0012,241.8667,0.0042
23,137.2889,0.0058,69.7111,0.0046,304.8222,5.3778,0.0349,30.6222,0.0252,35.1333,0.0004,30.6444,0.0014,241.9111,0.0043
24,138.4222,0.0057,70.2667,0.0045,307.7778,5.5111,0.0335,30.5333,0.0241,35.4444,0.0004,30.9333,0.0013,243.4222,0.0042
25,138.3778,0.0056,70.1778,0.0044,307.0,5.4444,0.0335,31.1333,0.0226,35.4889,0.0004,30.8222,0.0013,243.4,0.0042
26,136.7333,0.0057,69.1333,0.0043,304.3778,5.0444,0.0351,30.7778,0.0229,35.0889,0.0004,30.9333,0.0012,240.7778,0.0042
27,135.2444,0.0057,68.3333,0.0044,300.3556,4.8667,0.0366,31.1556,0.0234,34.5778,0.0004,30.8222,0.0013,237.7111,0.0042
28,135.4667,0.0057,68.4889,0.0044,301.4889,5.0222,0.0363,30.4444,0.0247,34.6222,0.0004,30.7111,0.0013,238.8,0.0042
29,133.4667,0.0058,67.5556,0.0046,297.1778,5.0,0.0382,30.4444,0.0277,34.2,0.0004,30.4,0.0019,234.7333,0.0043
30,133.6444,0.0058,67.7111,0.0047,297.3556,4.2667,0.0385,30.6222,0.027,34.1556,0.0004,30.5111,0.0019,235.5111,0.0044
31,133.8,0.0058,67.9333,0.0047,298.5778,3.8222,0.0384,31.3333,0.0269,34.3556,0.0004,31.1556,0.0019,235.0,0.0043
32,137.0222,0.0055,69.0222,0.0044,303.4222,4.6889,0.0352,30.6,0.0248,34.8222,0.0004,31.0,0.0013,240.0889,0.0042
33,137.8889,0.0054,69.5333,0.0045,305.9333,5.0222,0.0345,30.7111,0.0249,35.2667,0.0004,30.8222,0.0012,241.9111,0.0042
34,136.9778,0.0059,68.9778,0.0045,302.5111,5.3333,0.0356,29.8889,0.0249,34.8667,0.0004,30.2444,0.0012,239.9111,0.0043
35,136.4889,0.0058,69.0889,0.0047,303.0444,4.9111,0.0359,30.9111,0.0236,34.9111,0.0004,30.4667,0.0012,239.5556,0.0042
36,135.7778,0.0059,69.0,0.0048,302.4667,4.9778,0.036,30.8,0.0236,34.8222,0.0003,30.7333,0.0011,238.8889,0.0043
37,137.7111,0.0054,69.6667,0.0049,305.0667,4.9333,0.0348,31.4667,0.0245,35.0889,0.0004,30.7333,0.0013,241.6222,0.0042
38,138.2222,0.0054,70.0444,0.0047,306.1333,4.8222,0.0343,31.3111,0.0242,35.1778,0.0003,31.0889,0.0013,242.3778,0.0042
39,138.7778,0.0053,70.1556,0.0046,307.8,4.6889,0.0336,31.2889,0.0236,35.4,0.0003,31.4667,0.0013,243.4889,0.0041
40,138.0889,0.0052,69.8444,0.0046,307.2222,5.0,0.0337,30.7333,0.0229,35.3778,0.0004,31.2222,0.0013,243.0,0.0041
41,138.4,0.0052,69.7778,0.0048,306.9333,5.2889,0.0338,30.7778,0.0234,35.3333,0.0004,30.9333,0.0014,243.1333,0.0041
42,137.2444,0.0053,69.3778,0.0048,305.2444,5.2889,0.0345,30.7778,0.0236,35.1111,0.0003,30.5778,0.0015,241.0,0.0042
43,136.2,0.0054,68.6222,0.0047,302.9778,4.4667,0.0359,30.6222,0.0233,34.8444,0.0003,30.7556,0.0014,238.8667,0.0044
44,135.7111,0.0054,68.6444,0.0047,302.1333,4.5556,0.0363,30.7556,0.0223,34.8,0.0003,30.8667,0.0013,239.0889,0.0043
45,136.4889,0.0054,69.0667,0.0046,303.0667,4.6444,0.0359,30.7111,0.0226,34.9111,0.0003,30.8444,0.0012,240.1111,0.0042
46,137.1333,0.0055,69.3778,0.0048,303.7333,5.3111,0.0356,30.4667,0.025,34.9333,0.0004,30.5333,0.0013,240.6444,0.0042
47,137.9111,0.0055,69.6667,0.0046,304.8667,4.7556,0.035,31.3333,0.0245,35.0667,0.0004,30.7333,0.0012,241.0889,0.0041
48,137.7556,0.0055,69.4222,0.0046,306.4444,4.1556,0.0344,31.6444,0.0243,35.0889,0.0004,31.7111,0.0012,241.2889,0.0041
49,136.9778,0.0053,69.1111,0.0046,305.0222,3.6,0.035,32.3111,0.0226,34.9556,0.0004,32.1778,0.0013,240.4889,0.004
50,136.2222,0.0054,69.0222,0.0047,304.2222,3.6667,0.0356,31.7778,0.0242,34.7778,0.0004,32.1556,0.0013,239.9111,0.0041
51,136.4444,0.0055,69.0,0.0046,302.4222,4.7778,0.036,31.0444,0.0237,34.7778,0.0004,30.6444,0.0013,239.2,0.004
52,137.4667,0.0055,69.6889,0.0045,306.8889,4.3333,0.0345,31.9333,0.025,35.2667,0.0004,31.7111,0.0014,242.5556,0.004
53,137.8667,0.0054,69.6889,0.0045,306.5111,4.2444,0.0344,31.6444,0.0231,35.2889,0.0004,31.5556,0.0014,241.5778,0.0041
54,137.4,0.0054,69.4222,0.0047,306.5556,3.6889,0.0347,31.9333,0.0237,35.1111,0.0004,32.3111,0.0014,241.4889,0.0042
55,137.2444,0.0058,69.5111,0.0046,303.9778,4.9556,0.0351,30.5556,0.0222,35.0,0.0004,30.9556,0.0013,240.6444,0.0042
56,137.4444,0.0056,69.4444,0.0046,305.6222,5.2667,0.0344,30.8222,0.0229,35.1556,0.0004,30.8444,0.0013,242.0889,0.0042
57,137.7778,0.0056,69.9333,0.0044,305.8222,5.5111,0.0342,31.0667,0.0232,35.1778,0.0004,30.7778,0.0012,242.4222,0.0042
58,137.7778,0.0051,69.9556,0.0046,306.6667,5.3111,0.0338,31.1333,0.0221,35.2667,0.0003,30.7333,0.001,242.3778,0.0041
59,137.8,0.0052,69.9333,0.0047,307.0667,5.1333,0.0339,31.3556,0.0213,35.3333,0.0004,31.3111,0.001,242.5778,0.0042
60,137.3111,0.0053,69.4222,0.0048,305.1556,5.0222,0.0346,30.8889,0.0235,35.2222,0.0004,31.1556,0.0012,241.8,0.0042
61,137.0444,0.0055,69.4444,0.0048,305.0667,5.2444,0.0349,30.7111,0.0241,35.2,0.0004,31.0222,0.0013,241.2444,0.0042
62,136.9333,0.0055,69.2444,0.0047,302.9111,5.1333,0.0358,30.3778,0.0236,34.9778,0.0004,30.3778,0.0013,240.5111,0.0042
63,137.6889,0.0055,69.8,0.0047,305.3778,5.4,0.0345,30.8667,0.0221,35.2222,0.0004,30.7333,0.0012,242.3333,0.0042
64,138.3556,0.0053,70.0222,0.0045,307.3778,4.6444,0.0338,32.0444,0.0215,35.3556,0.0003,31.6444,0.0011,243.2889,0.004
65,138.1333,0.0052,69.8889,0.0044,308.6,4.3333,0.0334,32.4444,0.0222,35.3111,0.0004,32.4,0.0011,242.7778,0.004
66,137.4667,0.0053,69.6667,0.0044,306.3333,4.3333,0.0346,31.7333,0.0228,35.1778,0.0004,32.0667,0.0011,241.9556,0.004
67,136.7556,0.0053,69.2889,0.0046,304.2667,4.7111,0.0353,30.8667,0.0225,34.8889,0.0004,31.1556,0.0011,240.6444,0.004
68,137.6222,0.0054,69.6444,0.0046,305.6889,5.0444,0.0345,31.0222,0.0232,35.2444,0.0004,30.9111,0.0011,241.8667,0.004
69,133.9111,0.0056,67.8444,0.005,300.0444,5.0667,0.0374,30.0,0.0254,34.5778,0.0004,30.0,0.0013,236.3556,0.0041
70,133.6,0.0059,67.2667,0.005,296.2667,4.3111,0.0391,30.9556,0.0255,34.1333,0.0004,30.3556,0.0013,234.1333,0.0042
71,133.5556,0.0059,67.0667,0.0049,296.3111,3.9778,0.0394,30.3556,0.0251,34.0667,0.0004,30.9556,0.0014,234.6444,0.0042
72,134.2889,0.0068,67.3778,0.0047,297.1556,4.5556,0.0388,30.2,0.0297,34.1778,0.0005,30.5333,0.0014,234.8444,0.0041
73,132.0444,0.0083,66.5778,0.0049,294.8444,5.6444,0.0395,28.7778,0.0394,34.0667,0.0006,29.7556,0.0016,233.6667,0.0043
74,129.5778,0.0091,65.7111,0.0053,287.3111,6.9333,0.0428,27.4667,0.0441,33.2444,0.0008,27.6667,0.0016,228.5333,0.0044
75,129.8667,0.0089,66.1333,0.0054,286.9111,7.2667,0.0428,27.2889,0.0422,33.3556,0.0008,27.6444,0.0015,229.4889,0.0044
76,130.6889,0.0082,66.2444,0.0056,288.0444,7.4889,0.0422,27.1333,0.0368,33.2889,0.0006,27.2222,0.0014,229.2444,0.0044
77,130.8444,0.0082,66.6667,0.0055,288.9556,7.4222,0.0418,27.7111,0.0379,33.7333,0.0006,27.4222,0.0014,230.8222,0.0043
78,131.7556,0.0085,66.8222,0.0056,291.4222,5.6889,0.0413,29.2444,0.0331,33.6889,0.0007,28.7111,0.0012,231.6444,0.0043
79,132.3778,0.0084,67.3111,0.0053,292.8444,5.4889,0.0408,29.5556,0.033,33.9556,0.0007,29.1333,0.0012,233.2222,0.0042
80,132.2889,0.0088,67.0,0.0051,294.4222,4.8667,0.0402,29.6889,0.0343,33.8222,0.0007,29.8222,0.0013,232.9333,0.0041
81,130.9333,0.0101,66.2222,0.0052,290.5111,5.6889,0.0415,28.9556,0.0364,33.5111,0.0006,28.9111,0.0013,230.1333,0.0042
82,130.7333,0.0101,66.1778,0.0053,288.9556,5.9111,0.0425,28.2889,0.0377,33.5111,0.0007,28.5111,0.0014,229.9111,0.0044
83,131.0889,0.0098,66.3556,0.0052,290.2889,6.1778,0.0413,28.5111,0.0363,33.6,0.0006,28.4889,0.0013,230.2,0.0043
84,132.6,0.0098,67.3556,0.0049,293.2222,6.6,0.0401,28.8222,0.0366,34.0444,0.0008,28.5111,0.0013,233.8889,0.0044
85,131.3111,0.0101,66.8222,0.0048,292.7333,6.1778,0.0402,28.8222,0.0388,33.7556,0.0007,29.2444,0.0013,232.6222,0.0043
86,130.6889,0.0102,66.6,0.0051,289.6444,6.7333,0.0418,28.2,0.0389,33.5556,0.0008,28.0444,0.0013,230.7111,0.0043
87,130.5333,0.0094,66.4222,0.0054,289.1556,7.0889,0.0417,28.0,0.0421,33.5333,0.0006,27.9333,0.0017,230.2,0.0042
88,131.4667,0.0093,66.9111,0.0056,291.2,6.9778,0.0409,28.1778,0.0412,33.8222,0.0006,28.1111,0.0021,231.4,0.0042
89,130.2667,0.0094,65.7778,0.0059,289.8222,6.7556,0.0412,27.9111,0.0401,33.5556,0.0006,28.4889,0.0021,229.6,0.0042
@yuuki
Copy link
Copy Markdown
Author

yuuki commented Mar 17, 2023

ChatGPT (mode: GPT-4)の応答。

front-endサービスのlatencyが上昇しているので、各マイクロサービスのlatency, throughput, errorsのメトリクスを調べ、原因を特定します。

まず、front-endサービスのエラーが増加しているかどうかを確認します。
データによると、front-endのエラーは時間とともに変動していますが、全体的に増加傾向にはないようです。

次に、front-endが依存しているマイクロサービス(orders, carts, user, catalogue)のlatencyを調べます。

s-carts_latency: 時間とともに増加傾向
s-catalogue_latency: 増加傾向がありますが、他のマイクロサービスに比べて影響は小さい
s-orders_latency: 増加傾向
s-user_latency: 増加傾向
これらの結果から、carts, orders, userのマイクロサービスのlatencyが増加していることがわかります。また、front-endがordersに依存しているため、ordersの他の依存サービス(payment, shipping)も調べます。

s-payment_latency: 一定のままで大きな変動はない
s-shipping_latency: 増加傾向がありますが、他のマイクロサービスに比べて影響は小さい
最後に、throughputについても確認しますが、特に目立った変動はありません。

以上の分析から、front-endサービスのlatency上昇の原因は、carts, orders, userマイクロサービスのlatency増加によるものと考えられます。特にcartsとordersのlatencyが顕著に増加しているため、これらのマイクロサービスに問題がある可能性が高いです。これらのマイクロサービスを最初に調査・最適化することで、front-endサービスのlatencyを改善できるでしょう。

@yuuki
Copy link
Copy Markdown
Author

yuuki commented Mar 17, 2023

carts podにCPU過負荷をあたえたときのデータセットなので、正解はcartsサービス。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment