Logo
Hyunsu Blog
algorithm-tree

๐Ÿ“†Published :Aug 1, 2023 โ€ข

๐Ÿ“†Updated :Aug 1, 2023 โ€ข

โ˜•๏ธ4min

AWS Lambda, playwright, Slack์„ ํ™œ์šฉํ•œ ๋ฐฐ์†ก์ •๋ณด Tracker ๊ฐœ๋ฐœ๊ธฐ

1.๋ฐฐ๊ฒฝ

์†Œ๋ฏธ์—๋Š” ๋ฐฐ์†ก์ •๋ณด ์š”์ฒญํ•˜๊ธฐ๋ผ๋Š” ๊ธฐ๋Šฅ์ด ์žˆ๋‹ค. ์•„์ง๊นŒ์ง€ ์„œ๋น„์Šค์—์„  ๋ฐฐ์†ก์ •๋ณด๋ฅผ ๋ฏธ๋ฆฌ ์•Œ๋ ค์ฃผ์ง€ ์•Š๊ณ  ๋ฐฐ์†ก ์ •๋ณด ์š”์ฒญ์„ ์›ํ•˜๋Š” ์œ ์ €์—๊ฒŒ๋งŒ ๊ธฐ๋Šฅ์„ ์ œ๊ณตํ•˜๊ณ  ์žˆ๋‹ค.

Alt text

๊ธฐ๋Šฅ์ด ๊ณ ๋„ํ™” ๋˜์ง„ ์•Š์•„ ์–ด๋“œ๋ฏผ์ด ์š”์ฒญ ์ฒ˜๋ฆฌ๋ฅผ ํ•˜๊ธฐ ์œ„ํ•ด์„  ์ผ๋ จ์˜ ์ˆ˜๋™์ž‘์—…์„ ๊ฑฐ์น˜๊ฒŒ ๋œ๋‹ค.

1.์šด์˜์ž๋Š” ๋น ๋ฅด๊ฒŒ ํƒญ์„ ์ „ํ™˜ํ•˜์—ฌ ๋ฐฐ์†ก์ •๋ณด๋ฅผ ํ™•์ธํ•˜๋Ÿฌ ํŠน์ • ์‚ฌ์ดํŠธ์— ๋“ค์–ด๊ฐ„๋‹ค.

2.๋กœ๊ทธ์ธ์„ ์‹œ๋„ํ•œ๋‹ค.

3.๋ฐฐ์†ก ์ •๋ณด๋ฅผ ์š”์ฒญํ•œ ์ฃผ๋ฌธ ์ •๋ณด๋“ค์„ ํ•˜๋‚˜์”ฉ ์ฐพ์•„๋ณธ๋‹ค. ์ฃผ๋ฌธํ•œ์ง€ ๊ฝค๋‚˜ ์˜ค๋ž˜ ๋œ ๊ฒƒ์ด๋ผ๋ฉด ํŽ˜์ด์ง€๋ฅผ ๋„˜๊ฒจ ๋’ค๋กœ ๊ฐ€์•ผ ํ•œ๋‹ค. ํ˜น์€ ๋ฌดํ•œ ์Šคํฌ๋กค๋กœ UI๊ฐ€ ๋˜์–ด ์žˆ๋‹ค๋ฉด ์Šคํฌ๋กค์„ ๊ณ„์† ๋‚ด๋ฆฌ๊ณ  ์žˆ์„ ๊ฒƒ์ด๋‹ค.

  1. ์ •๋ณด๋ฅผ ์ฐพ์•˜๋‹ค๋ฉด ํ…์ŠคํŠธ๋ฅผ ๋“œ๋ž˜๊ทธ ํ•˜์—ฌ ๋ณต์‚ฌํ•˜๊ณ 
  2. ๋‹ค์‹œ ์–ด๋“œ๋ฏผ ํƒญ์œผ๋กœ ๋Œ์•„์™€ ์šด์†ก์žฅ ๋ฒˆํ˜ธ๋ฅผ ๋ถ™์—ฌ ๋„ฃ๊ธฐ ํ•˜๊ณ  ํƒ๋ฐฐ์‚ฌ๋ฅผ ์„ ํƒํ•œ ํ›„ ๋ฐฐ์†ก ์ •๋ณด ์ „์†ก ๋ฒ„ํŠผ์„ ๋ˆ„๋ฆ„์œผ๋กœ์จ ๋น„๋กœ์†Œ ์ž‘์—…์ด ์™„๋ฃŒ๋œ๋‹ค.

์ด ๊ณผ์ •์ด ์ƒ๊ฐ ๋ณด๋‹ค ์—๋„ˆ์ง€๋ฅผ ๋งŽ์ด ์†Œ๋ชจ ํ•˜๊ฒŒ ํ•˜๋Š” ์ผ์ด๋‹ค.

๋งฅํ‚จ์ง€ ์กฐ์‚ฌ์— ๋”ฐ๋ฅด๋ฉด, ์ง์žฅ์ธ์€ ํ•˜๋ฃจ ํ‰๊ท  2.2์‹œ๊ฐ„ ์ด์ƒ ๋ฉ”์‹ ์ €๋‚˜ ์ด๋ฉ”์ผ ์•Œ๋ฆผ์— ๋ฐฉํ•ด ๋ฐ›๋Š”๋‹ค๊ณ  ๋‚˜ํƒ€๋‚ฌ๋‹ค '๋”ฅ์›Œํฌ(Deep Work)'์˜ ์ €์ž์ธ ์นผ ๋‰ดํฌํŠธ(Cal Newport)๋„ ์ด๋ฉ”์ผ, ๋ฉ”์‹ ์ €์™€ ๊ฐ™์€ ํ”ผ์ƒ์ ์ธ ์—…๋ฌด์™€ ์ปจํ…์ŠคํŠธ ์Šค์œ„์นญ(Context-Switching)์„ ์ค„์ด๋Š” ์ผ์ด ์—…๋ฌด ๋ชฐ์ž…์— ๊ฐ€์žฅ ์ค‘์š”ํ•˜๋‹ค๊ณ  ๋ฐํ˜”๋‹ค. ์‹ค์ œ ์›ํŽ˜์ด์ง€ ํ˜‘์—…ํˆด '๋…ธ์…˜'์˜ ์กฐ์‚ฌ ๊ฒฐ๊ณผ์— ๋”ฐ๋ฅด๋ฉด, ์ด์šฉ์ž ์ค‘ 98%๊ฐ€ ๋…ธ์…˜์œผ๋กœ ์—…๋ฌด ์‹œ๊ฐ„์„ ์ ˆ์•ฝํ•œ ๊ฒƒ์œผ๋กœ ๋‚˜ํƒ€๋‚ฌ๋‹ค. ๋˜ ํ”„๋กœ์ ํŠธ ์†Œ์š” ์‹œ๊ฐ„ ํ‰๊ท  32%๋ฅผ ๋‹จ์ถ• ์‹œํ‚ค๊ณ  ์ด๋ฉ”์ผ ์ˆ˜์‹  ๋ฐ ๋ฐœ์‹ ๋Ÿ‰๋„ 26% ์ค„์ด๋Š” ํšจ๊ณผ๋ฅผ ๊ฑฐ๋’€๋‹ค๊ณ  ๋‹ตํ–ˆ๋‹ค.

์‹ฌ์ง€์–ด ์•„์ง ์†ก์žฅ๋ฒˆํ˜ธ๊ฐ€ ์ค€๋น„๋˜์ง€ ์•Š์€ ๋ฐฐ์†ก์ค€๋น„์ธ ๊ฒฝ์šฐ, ๋‹ค์Œ๋‚  ๋‹ค์‹œ ํ™•์ธํ•ด์„œ ๋ฒˆํ˜ธ๋ฅผ ์ค˜์•ผ ํ•œ๋‹ค.

๋ฐฐ์†ก ์•Œ๋ฆผ ๊ธฐ๋Šฅ ์Šคํ”„๋ฆฐํŠธ๋ฅผ ์‹œ์ž‘ํ•˜๋Š” ๊ฒƒ๋„ ๋ฐฉ๋ฒ• ์ด๋‹ค. ์ƒˆ๋กœ์šด ๋ฐฐ์†ก ์ •๋ณด ์•Œ๋ฆผ ๊ธฐ๋Šฅ์„ ์™ธ๋ถ€ API๋ฅผ ํ†ตํ•ด ๋„์ž…ํ•˜๋Š” ๊ฒƒ์€ ๊ฐ€๋Šฅํ•˜์ง€๋งŒ, ์ด ๊ณผ์ •์— ์—ฌ๋Ÿฌ ๋‹ด๋‹น์ž๊ฐ€ ๊ด€์—ฌํ•˜๊ฒŒ ๋˜๋ฉด์„œ ์Šคํ”„๋ฆฐํŠธ์˜ ์‹ ์†ํ•œ ์ง„ํ–‰์ด ์–ด๋ ค์›Œ์งˆ ์ˆ˜ ์žˆ๋‹ค. ์ด๋Ÿฌํ•œ ๊ธฐ๋Šฅ ์ถ”๊ฐ€๋Š” ํŒ€ ๋‚ด์—์„œ ์ฑ…์ž„์ด ํ™•๋Œ€๋˜๋ฉฐ, ์—ฌ๋Ÿฌ ์ดํ•ด๊ด€๊ณ„์™€ ์กฐ์œจ ๊ณผ์ •์ด ํ•„์š”ํ•ด์ง€๊ธฐ ๋•Œ๋ฌธ์— ์‹ ์†๋ฉด์—์„œ๋Š” ๋–จ์–ด์ง„๋‹ค.

๋น ๋ฅด๊ณ  ๊ฐ€๋ณ๊ฒŒ ์ƒ์‚ฐ์„ฑ์„ ์ฆ๋Œ€ ์‹œํ‚ฌ ๋ฐฉ๋ฒ•์œผ๋กœ ์Šฌ๋ž™์œผ๋กœ ๋งค์ผ ์˜ค์ „ ์ •ํ•ด์ง„ ์‹œ๊ฐ„์— ๋ด‡์œผ๋กœ ์•Œ๋ ค์ฃผ๋Š” ๋ฐฉ์‹์„ ์„ ํƒํ–ˆ๋‹ค. ์Šฌ๋ž™์€ ๋ชจ๋“  ํŒ€์›๋“ค์˜ ์ค‘์•™ ์‹œ์Šคํ…œ์ด๋ฏ€๋กœ, ์Šฌ๋ž™์ด ์—…๋ฌด์‹œ ์†Œํ”„ํŠธ์›จ์–ด ํˆด ์ปจํ…์ŠคํŠธ ์Šค์œ„์นญ์—๋Š” ๋„์›€์ด ๋  ๊ฒƒ์ด๋ผ ์ƒ๊ฐํ–ˆ๋‹ค.

2.ํˆด ์„ ํƒ

AWS Lambda ์„ ํƒ ์ด์œ ์™€ ํŠธ๋ ˆ์ด๋“œ ์˜คํ”„

AWS Lambda๋ฅผ ์„ ํƒํ•œ ์ฃผ๋œ ์ด์œ ๋Š” ๋ฌด์—‡๋ณด๋‹ค๋„ "์„œ๋ฒ„๋ฆฌ์Šค(Serverless)" ์•„ํ‚คํ…์ฒ˜์˜ ์ด์  ์ด๋‹ค. ์ƒ์‚ฐ์„ฑ์„ ์œ„ํ•œ ๊ฐ€๋ฒผ์šด ํˆด์„ ๋ฐฑ์—”๋“œ ์„œ๋ฒ„์— ์˜์กดํ•˜์—ฌ ๊ฐ™์ด ๋ณ‘ํ–‰ํ•˜๊ธฐ ๋ณด๋‹ค ๋…๋ฆฝ์ ์œผ๋กœ ๊ฐ€์ ธ ๊ฐ€๋Š”๊ฒŒ ์œ ์ง€๋ณด์ˆ˜๊ฐ€ ๋” ์ข‹์„ ๊ฒƒ์ด๋ผ ์ƒ๊ฐํ–ˆ๊ณ , ๋งŒ์•ฝ ์„œ๋น„์Šค ๊ณ ๋„ํ™”๋กœ ์ด ๊ธฐ๋Šฅ์ด ์‚ฌ๋ผ์งˆ ์ˆ˜๋„ ์žˆ๊ธฐ ๋•Œ๋ฌธ์— Lambda๋ฅผ ์‚ฌ์šฉํ•˜๋ฉด ์„œ๋ฒ„ ๊ด€๋ฆฌ์˜ ๋ถ€๋‹ด ์—†์ด ์ฝ”๋“œ๋ฅผ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ๋‹ค. ์ด๋Š” ํŠนํžˆ ์šฐ๋ฆฌ์™€ ๊ฐ™์ด ๊ฐ€๋ณ€์ ์ธ ํŠธ๋ž˜ํ”ฝ์„ ์ฒ˜๋ฆฌํ•˜๋Š” ์„œ๋น„์Šค์— ์ ํ•ฉํ•˜๋ฉฐ ํ•˜๋ฃจ์— ํ•œ๋ฒˆ ํ˜น์€ ๋‘๋ฒˆ์˜ ํ˜ธ์ถœ์ด ์ƒ๊ธฐ๋‹ˆ ๋น„์šฉ์ ์œผ๋กœ๋„ ๋ถ€๋‹ด์ด ๋˜์ง€ ์•Š๋Š”๋‹ค๊ณ  ์ƒ๊ฐํ–ˆ๋‹ค.

๋ฌผ๋ก , ํŠธ๋ ˆ์ด๋“œ ์˜คํ”„๋„ ์กด์žฌ ํ•œ๋‹ค. Lambda์˜ ๊ฒฝ์šฐ, ์‹คํ–‰ ์‹œ๊ฐ„, ๋ฉ”๋ชจ๋ฆฌ ๋“ฑ์— ์ œํ•œ์ด ์žˆ์–ด์„œ, ๋” ๋ณต์žกํ•˜๊ณ  ์ž์› ์ง‘์•ฝ์ ์ธ ์ž‘์—…์—๋Š” ์ ํ•ฉํ•˜์ง€ ์•Š์„ ์ˆ˜ ์žˆ๊ณ , ๋˜ํ•œ "์ฝœ๋“œ ์Šคํƒ€ํŠธ" ๋ฌธ์ œ๋กœ ์ดˆ๊ธฐ ์‘๋‹ต ์‹œ๊ฐ„์ด ๊ธธ์–ด์งˆ ์ˆ˜ ์žˆ๋‹ค๋Š” ์ ์„ ๋ฆฌ์„œ์น˜๋ฅผ ํ†ตํ•ด ์•Œ๊ฒŒ ๋˜์—ˆ๋Š”๋ฐ ์‚ฌ์‹ค ์ด ๋ถ€๋ถ„์— ๋Œ€ํ•ด์„œ๋Š” ์ž์„ธํžˆ ๋“ค์—ฌ๋‹ค ๋ณด์งˆ ๋ชปํ–ˆ๋‹ค. ์Šฌ๋ž™์—๋„ ๋™์ž‘์ด ๋œ ๊ฑธ ํ™•์ธ ํ•œ ์ดํ›„๋กœ๋Š” ๋‹ค์‹œ ํ™•์ธํ•ด ๋ณผ ๊ธฐํšŒ๊ฐ€ ์ž˜ ์—†์—ˆ๋‹ค.๐Ÿ˜“

Playwright ์„ ํƒ์ด์œ 

playwright๋Š” ์ด์ „ ํšŒ์‚ฌ์—์„œ E2Eํ…Œ์ŠคํŠธ ์ž‘์„ฑ์„ ํ•  ๋•Œ ์‚ฌ์šฉํ–ˆ๋˜ ๋„๊ตฌ๋กœ ๊ทธ ๋•Œ ๋‹น์‹œ ํŒ€ํšŒ์˜๋ฅผ ํ†ตํ•ด puppeteer์™€ ๋น„๊ตํ•˜์—ฌ playwright์˜ ์ด์ ์„ ์–ด๋Š์ •๋„๋Š” ์ธ์ง€ ํ•˜๊ณ  ์žˆ์—ˆ๋‹ค.

๋„๊ตฌ๋“ค ์ค‘ ํฐ ์ฐจ์ด๊ฐ€ ์—†๋‹ค๋ฉด ๊ทธ๋‚˜๋งˆ ์ต์ˆ™ํ•œ ๋„๊ตฌ๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๊ฒƒ์ด First Time To Hello ๋ฅผ ๋ณด๊ธฐ์—” ์ œ๊ฒฉ์ด๋ผ๊ณ  ์ƒ๊ฐํ–ˆ๊ณ  ์ผ๋‹จ ๋น ๋ฅด๊ฒŒ ๋งŒ๋“ ๋‹ค ๋ผ๋Š” ๋ชฉํ‘œ์— ์ง‘์ค‘ํ•ด์„œ ์„ ํƒํ–ˆ๋‹ค.

playwright๋Š” puppeteer๋ณด๋‹ค developer experience๊ฐ€ ๋” ์ข‹๋‹ค๊ณ  ์ฃผ์žฅํ–ˆ์—ˆ๋‹ค.

ํŠนํžˆ ์ž๋™ ์ง€์—ฐ ์ฒ˜๋ฆฌ๋ถ€๋ถ„์„ ์ ์šฉํ•˜๊ณ  ๊ฐœ์„ ํ–ˆ๋‹ค๊ณ  ํ–ˆ์—ˆ๋‹ค.

์˜ˆ์ „ puppeteer๋ฅผ ์‚ฌ์šฉํ•  ๋‹น์‹œ ์œ ์ €์˜ ์•ก์…˜์ฒ˜๋ฆฌ ๊ณผ์ •์—์„œ ๋“œ๋ž๋˜๋Š” ํ˜„์ƒ๋“ค์ด ๋นˆ๋ฒˆํ•˜์—ฌ ์ด๋Ÿด ๋• wait() ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ฌผ๋ฆฌ์ ์œผ๋กœ ์ง€์—ฐ์„ ์ฃผ๊ณค ํ–ˆ๋Š”๋ฐ, ์ง€์—ฐ์ด ๋งŽ์„ ์ˆ˜๋ก ์„ฑ๋Šฅ์ ์œผ๋กœ ์ข‹์ง€ ์•Š์€ ๊ฑด ๋‹น์—ฐํ•˜๋‹ค.์ด๋Ÿฐ ๋ถ€๋ถ„์„ playwright์—์„œ๋Š” ๊ฐœ์„ ํ–ˆ๋‹ค๊ณ  ํ•œ๋‹ค.

3. AWS Lambda์˜ ๊ตฌ์กฐ ๊ฐ„๋žต ์„ค๋ช…

AWS Lambda๋Š” ์ด๋ฒคํŠธ ๊ธฐ๋ฐ˜, ์„œ๋ฒ„๋ฆฌ์Šค ์ปดํ“จํŒ… ์„œ๋น„์Šค ์ด๋‹ค. ๊ฐ„๋‹จํžˆ ๋งํ•ด์„œ, ํŠน์ • ์ด๋ฒคํŠธ๊ฐ€ ๋ฐœ์ƒํ•˜๋ฉด AWS์—์„œ ์ฝ”๋“œ๋ฅผ ์ž๋™์œผ๋กœ ์‹คํ–‰ํ•ด์ฃผ๋Š” ์„œ๋น„์Šค์ด๋‹ค. Lambda ํ•จ์ˆ˜๋Š” ๋…๋ฆฝ์ ์œผ๋กœ ์‹คํ–‰๋˜๋ฉฐ, ํ•„์š”ํ•œ ๋ชจ๋“  ๊ฒƒ์€ AWS๊ฐ€ ๊ด€๋ฆฌ ํ•œ๋‹ค. ์ด๋Š” ๊ฐœ๋ฐœ์ž๋กœ ํ•˜์—ฌ๊ธˆ ์ธํ”„๋ผ ๊ด€๋ฆฌ๋ณด๋‹ค ๋น„์ฆˆ๋‹ˆ์Šค ๋กœ์ง ๊ฐœ๋ฐœ์— ๋” ์ง‘์ค‘ํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ค€๋‹ค. aws ์˜์ƒ๋“ค์„ ๋ณด๋ฉด ์ด ๋ถ€๋ถ„์„ ๊ฝค๋‚˜ ๊ฐ•์กฐ ํ•˜๊ณ  ์žˆ๋‹ค.

์•„๋ž˜ ์ด๋ฏธ์ง€๋Š” ํ˜„์žฌ ์ ์šฉ๋œ aws lambda ํŽ˜์ด์ง€์˜ ๊ธฐ๋ณธ์ ์ธ ์ „์ฒด๊ตฌ์กฐ ๋ฐ ํ๋ฆ„์ด๋‹ค.

Alt text

  1. API Gateway: Amazon API Gateway๋Š” ํด๋ผ์ด์–ธํŠธ์™€ ๋ฐฑ์—”๋“œ ์‚ฌ์ด์˜ โ€œ๋ฌธโ€๊ณผ ๊ฐ™์€ ์—ญํ• ์„ ํ•œ๋‹ค. API Gateway์— ์ •์˜๋œ URL๋กœ HTTP ์š”์ฒญ์„ ๋ณด๋‚ด๋ฉด, API GateWay๊ฐ€ ์ด ์š”์ฒญ์„ ๋ฐ›์•„ Lambdaํ•จ์ˆ˜๋ฅผ ํ˜ธ์ถœํ•˜๊ณ , Lambdaํ•จ์ˆ˜์˜ ์ฒ˜๋ฆฌ๊ฒฐ๊ณผ๋ฅผ ๋‹ค์‹œ API Gateway๋ฅผ ํ†ตํ•ด ํด๋ผ์ด์–ธํŠธ์—๊ฒŒ ์‘๋‹ตํ•  ์ˆ˜ ์žˆ๋‹ค. ์ดHTTP์š”์ฒญ์—” query paramerter ๋ฅผ ์ด์šฉํ•˜์—ฌ ์š”์ฒญ๋„ ๊ฐ€๋Šฅํ•˜๋‹ค.
  2. Lambda ํ•จ์ˆ˜ ํ˜ธ์ถœ: API Gateway๋Š” ์š”์ฒญ์— ๋งž๋Š” Lambda ํ•จ์ˆ˜๋ฅผ ํ˜ธ์ถœ ํ•œ๋‹ค.
  3. EventBridge ์ด๋ฒคํŠธ ์ฒ˜๋ฆฌ: EventBridge๋Š” ๋‹ค์–‘ํ•œ ์ด๋ฒคํŠธ๋ฅผ ๊ฐ์ง€ํ•˜๊ณ , ์ด์— ๋ฐ˜์‘ํ•˜์—ฌ Lambda ํ•จ์ˆ˜๋ฅผ ํŠธ๋ฆฌ๊ฑฐํ•  ์ˆ˜ ์žˆ๊ฒŒ ํ•ด์ค€๋‹ค.
  4. Cron Job ์Šค์ผ€์ค„๋ง: CloudWatch Events์˜ Cron ํ‘œํ˜„์‹์„ ์‚ฌ์šฉํ•˜๋ฉด ์ •ํ•ด์ง„ ์‹œ๊ฐ„ ์˜ˆ๋ฅผ ๋“ค์–ด ์•„์นจ 9์‹œ์— Lambdaํ•จ์ˆ˜๋ฅผ ์ž๋™์œผ๋กœ ํ˜ธ์ถœ ํ•  ์ˆ˜ ์žˆ๋‹ค. ์ด๋ฅผ ํ†ตํ•ด ์ฃผ๊ธฐ์ ์ธ ๋ฐ์ดํ„ฐ ์ฒ˜๋ฆฌ ๋“ฑ์˜ ์ž‘์—…์„ ์ž๋™ํ™”ํ•  ์ˆ˜ ์žˆ๋‹ค.
  5. Lambda ํ•จ์ˆ˜ ์‹คํ–‰: Lambda ํ•จ์ˆ˜๊ฐ€ ์‹คํ–‰๋˜์–ด ํ•„์š”ํ•œ ์ž‘์—…์„ ์ˆ˜ํ–‰ํ•˜๊ณ  ๊ฒฐ๊ณผ๋ฅผ ๋ฐ˜ํ™˜ ํ•œ๋‹ค.
  6. ๊ฒฐ๊ณผ ์ฒ˜๋ฆฌ: API Gateway๋ฅผ ํ†ตํ•ด ํด๋ผ์ด์–ธํŠธ์—๊ฒŒ ๊ฒฐ๊ณผ๋ฅผ ์‘๋‹ตํ•œ๋‹ค. ์ด ํ”„๋กœ์ ํŠธ์—์„œ๋Š” ์Šฌ๋ž™์œผ๋กœ ๊ฒฐ๊ณผ๋ฅผ ์‘๋‹ตํ•œ๋‹ค.

ํ˜ธ์ถœ ์ง„ํ–‰ ๊ณผ์ •

๋‹ค์Œ์˜ ์‹œํ€€์Šค ๋‹ค์ด์–ด๊ทธ๋žจ์— ๊ทธ๋ ค์ง„ ์ˆœ์„œ์— ์˜ํ•œ ํ๋ฆ„์œผ๋กœ ์ง„ํ–‰๋  ๊ฒƒ์ด๋‹ค. ์š”์ ์„ ๊ฐ„๋žตํ•˜๊ฒŒ ์ด์•ผ๊ธฐ ํ•˜์ž๋ฉด, Alt text

  1. ์ด๋ฒคํŠธ ํŠธ๋ฆฌ๊ฑฐ: ๋งค์ผ ์•„์นจ๋งˆ๋‹ค ํŠน์ • ์‹œ๊ฐ„์— Lambda ํ•จ์ˆ˜๋ฅผ ํŠธ๋ฆฌ๊ฑฐํ•˜๊ธฐ ์œ„ํ•ด AWS CloudWatch Events๋ฅผ ์‚ฌ์šฉํ•œ๋‹ค. ์ด๋ ‡๊ฒŒ ์„ค์ •ํ•˜๋ฉด, ์ •ํ•ด์ง„ ์‹œ๊ฐ„์— ์ž๋™์œผ๋กœ Lambda ํ•จ์ˆ˜๊ฐ€ ํ˜ธ์ถœ๋œ๋‹ค.
  2. ๋ฐ์ดํ„ฐ ์ฒ˜๋ฆฌ: ํ˜ธ์ถœ๋œ Lambda ํ•จ์ˆ˜๋Š” Playwright๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋„ค์ด๋ฒ„ ์ฃผ๋ฌธ ๋ฆฌ์ŠคํŠธ์— ์ ‘๊ทผํ•œ๋‹ค. ์—ฌ๊ธฐ์„œ ํ•„์š”ํ•œ ๋ฐฐ์†ก ์ •๋ณด๋ฅผ ์ถ”์ถœํ•œ๋‹ค.
  3. ์Šฌ๋ž™ ์ฑ—๋ด‡ ํ†ต์‹ : ์ถ”์ถœ๋œ ๋ฐฐ์†ก ์ •๋ณด๋Š” ์Šฌ๋ž™ API๋ฅผ ํ†ตํ•ด ์Šฌ๋ž™ ์ฑ—๋ด‡์œผ๋กœ ์ „์†ก๋œ๋‹ค. ์ฑ—๋ด‡์€ ์ด ์ •๋ณด๋ฅผ ์ ์ ˆํ•œ ํฌ๋งท์œผ๋กœ ๊ฐ€๊ณตํ•˜์—ฌ ์Šฌ๋ž™ ์ฑ„๋„์— ๋ฉ”์‹œ์ง€๋ฅผ ๋ณด๋‚ธ๋‹ค.

๋‹ค์Œ์€ AWS ์ ์šฉ์‹œ ์ฐธ๊ณ  ํ•˜๋ฉด ์ข‹์„ ์‚ฌํ•ญ์ด๋‹ค.

ํ•จ์ˆ˜์—๋„ ์ œํ•œ์ด ์žˆ๋‹ค. 15๋ถ„, ๊ทธ๋ฆฌ๊ณ  ๋ฉ”๋ชจ๋ฆฌ ํ• ๋‹น๋Ÿ‰๋„ 10GB๊นŒ์ง€ ๋‚˜ํƒ€๋‚ธ๋‹ค.

lambda-aws

์ ์šฉํ•˜๋Š” ๊ณผ์ •์—์„œ ๋งˆ์ฃผํ•œ ์ด์Šˆ๋“ค๋„ ์žˆ์—ˆ๋‹ค.

4 ์ง„ํ–‰ํ•˜๋ฉด์„œ ๊ฒช์€ ์ด์Šˆ

AWS Lambda ํ™˜๊ฒฝ ์—์„œ์˜ Playwright ์‚ฌ์šฉ ์ด์Šˆ

๋น ๋ฅด๊ฒŒ ์ฝ”๋“œ๋ฅผ ์ž‘์„ฑํ•œ ํ›„, zipํŒŒ์ผ๋กœ ๋ฌถ์–ด aws gui๋ฅผ ํ†ตํ•ด ์—…๋กœ๋“œ ํ•œ ๊ฒฐ๊ณผ๋Š” ์‹คํŒจ์˜€๋‹ค.

{ "errorType": "Error", "errorMessage": "browserType.launch: Executable doesn't exist at /home/sbx_user1051/.cache/ms-playwright/chromium-1067/chrome-linux/chrome\nโ•”โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•—\nโ•‘ Looks like Playwright Test or Playwright was just installed or updated. โ•‘\nโ•‘ Please run the following command to download new browsers: โ•‘\nโ•‘ โ•‘\nโ•‘ npx playwright install โ•‘\nโ•‘ โ•‘\nโ•‘ <3 Playwright Team โ•‘\nโ•šโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•", "trace": [ "browserType.launch: Executable doesn't exist at /home/sbx_user1051/.cache/ms-playwright/chromium-1067/chrome-linux/chrome", "โ•”โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•—", "โ•‘ Looks like Playwright Test or Playwright was just installed or updated. โ•‘", "โ•‘ Please run the following command to download new browsers: โ•‘", "โ•‘ โ•‘", "โ•‘ npx playwright install โ•‘", "โ•‘ โ•‘", "โ•‘ <3 Playwright Team โ•‘", "โ•šโ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•โ•", " at initCrawler (file:///var/task/index.js:21:34)", " at Runtime.handler (file:///var/task/index.js:205:30)", " at Runtime.handleOnceNonStreaming (file:///var/runtime/index.mjs:1086:29)" ] }

์—๋Ÿฌ๊ฐ€ ๋‚ฌ์„ ๋•Œ

๋กœ์ปฌํ™˜๊ฒฝ์—์„œ์˜ playwright๋ฅผ ์‚ฌ์šฉํ•  ๋•Œ๋Š” playwright๋ฅผ ์„ค์น˜ ํ•  ๋•Œ, headless browser ๋™์ž‘์„ ์œ„ํ•ด ์ž๋™์œผ๋กœ Chrome, Firefox, Webkit ๋“ฑ๊ณผ ๊ฐ™์€ ํ•„์š”ํ•œ ๋ธŒ๋ผ์šฐ์ €๋“ค์„ ๋‹ค์šด๋ฐ›๊ณ  ์„ค์น˜ํ•œ๋‹ค. ์ด ๊ณผ์ •์—์„œ ํ•„์š”ํ•œ ๋ธŒ๋ผ์šฐ์ € ์‹คํ–‰ ํŒŒ์ผ๋“ค์ด ๋กœ์ปฌ ์‹œ์Šคํ…œ์— ์ €์žฅ๋˜๋Š”๋ฐ, Lambda์˜ ์„œ๋ฒ„๋ฆฌ์Šค ์ปดํ“จํŒ… ํ™˜๊ฒฝ์—์„œ๋Š” Playwright๊ฐ€ ํ•„์š”๋กœ ํ•˜๋Š” ๋ธŒ๋ผ์šฐ์ € ์‹คํ–‰ํŒŒ์ผ์ด ํฌํ•จ๋˜์–ด ์žˆ์ง€ ์•Š๋‹ค.

์ด๋Š” AWS Lambda์˜ stateless ํŠน์„ฑ๊ณผ๋„ ์—ฐ๊ด€์ด ์žˆ๋Š”๋ฐ, ๋žŒ๋‹คํ•จ์ˆ˜๊ฐ€ ์ด๋ฒคํŠธ์— ์˜ํ•ด ํŠธ๋ฆฌ๊ฑฐ ๋  ๋•Œ๋งˆ๋‹ค ์™„์ „ํžˆ ์ƒˆ๋กœ์šด ํ™˜๊ฒฝ์—์„œ ํ˜ธ์ถœ๋œ๋‹ค.

๊ฒฐ๊ตญ AWS Lambdaํ™˜๊ฒฝ์—์„œ Playwright๊ฐ€ headless browser ๋™์ž‘์„ ํ•˜๊ธฐ ์œ„ํ•ด ๋ธŒ๋ผ์šฐ์ € ๋ฐ”์ด๋„ˆ๋ฆฌ๊ฐ€ ํ•„์š”ํ•˜๋‹ค.

puppeteer ๋ฅผ ์ฐธ๊ณ ํ•œ ์ž๋ฃŒ์—์„œ ์ฐพ์„ ์ˆ˜ ์žˆ์—ˆ๋Š”๋ฐ, chrome-aws-lambda ๋ฅผ ์ด์šฉํ•˜๋Š” ๊ฒƒ์ด๋‹ค.

chrome-aws-lambda ๋Š” ์›น๋ธŒ๋ผ์šฐ์ €๋ฅผ ์‹คํ–‰ํ•˜๋Š”๋ฐ ํ•„์š”ํ•œ ๋ฐ”์ด๋„ˆ๋ฆฌ๋กœ chrome ๋ฐ”์ด๋„ˆ๋ฆฌ์ด๋‹ค.

lambda playwright

๊ทธ๋ฆฌ๊ณ  Memory๋„ 512MB๋กœ ์žฌํ• ๋‹น ํ•ด์ค€๋‹ค. ๋ฉ”๋ชจ๋ฆฌ ์„ค์ •์€ lambda โ†’ configuration์—์„œ ๊ฐ€๋Šฅํ•˜๋‹ค.

Untitled

์ด์ œ ์ฝ”๋“œ๋กœ ๋Œ์•„๊ฐ€์„œ ๋‹ค์‹œ ๋””ํŽœ๋˜์‹œ๋ฅผ ์„ค์น˜ํ•œ ํ›„ ๋ผˆ๋Œ€ ์ฝ”๋“œ ๋ฒ ์ด์Šค๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™๋‹ค.

const express = require("express"); const chromium = require("chrome-aws-lambda"); //โœ… chrome binary const playwright = require("playwright-core"); //โœ… playwright-core, โŒplaywright const { WebClient } = require("@slack/web-api"); //โœ… slack api const { initCrawler } = require("./service"); const app = express(); const port = 3000; //http const http = require("http"); const server = http.createServer(app); const isDev = process.env.NODE_ENV !== "production"; // env const SLACK_BOT_TOKEN = process.env.SLACK_BOT_TOKEN; const SLACK_CHANNEL = process.env.SLACK_CHANNEL; const NAVER_ID = process.env.NAVER_ID; const NAVER_PW = process.env.NAVER_PW; const naverLoginUrl = "https://nid.naver.com/nidlogin.login?mode=form&url=https://www.naver.com"; const slack = new WebClient(SLACK_BOT_TOKEN); //create lambda function โœ…์‹ค์ œ ๋žŒ๋‹คํ•จ์ˆ˜๊ฐ€ ํ˜ธ์ถœ์„ ๋‹ด๋‹นํ•˜๋Š” handler ํ•จ์ˆ˜์ด๋‹ค. exports.handler = async (event, context, callback) => { // โœ… playwright-core ์— ์žˆ๋Š” chromium ๋ธŒ๋ผ์šฐ์ € ์—”์ง„์„ ์‹œ์ž‘ํ•œ๋‹ค. const browser = await playwright.chromium.launch({ args: chromium.args, executablePath: await chromium.executablePath, headless: chromium.headless, // ์ด ๊ตฌ๊ฐ„์ด headless์—ฌ์•ผ ์‹ค์ œ ์ฐฝ์ด ๋œจ์ง€ ์•Š๋Š”๋‹ค(GUI๋ฅผ ์œ„ํ•œ ๋ฉ”๋ชจ๋ฆฌ๊ฐ€ ๋” ํ•„์š”ํ•˜๊ธฐ ๋•Œ๋ฌธ) }); let shippingOrders = []; try { shippingOrders = await initCrawler(browser); // shippingOrders์˜ ์ฒซ๋ฒˆ์งธ ์ฃผ๋ฌธ ๋ฒˆํ˜ธ json์— ๋‹ด๊ธฐ }catch(){} }

popup์ฐฝ ๋‚ด์˜ element์— ์žˆ๋Š” ๋ฐฐ์†ก์ƒํƒœ ํ…์ŠคํŠธ๋ฅผ ๊ฐ€์ ธ์˜ค๋Š”๊ฒŒ ๊ฐ€๋Šฅํ• ๊นŒ?

ํ•˜๋‚˜์˜ ์ฃผ๋ฌธ๋ฒˆํ˜ธ์— ๋‹ค๋ฅธ ์ƒ์ ์—์„œ ๊ตฌ๋งคํ•œ ์—ฌ๋Ÿฌ๊ฐœ์˜ ์•„์ดํ…œ์˜ ๋ฐฐ์†ก์ •๋ณด๋“ค์ด popup์œผ๋กœ ๋˜์–ด์žˆ๋Š” ๊ฒฝ์šฐ, ๊ฐ๊ฐ์˜ ๋ชจ๋“  ์•„์ดํ…œ๋“ค์˜ popup ๋‚ด element๋ฅผ ์ฝ์–ด์„œ ๋ฐฐ์†ก์ƒํƒœ ๋ฅผ ๊ฐ€์ ธ์™€์•ผ ํ–ˆ๋‹ค. ๋‹ค๋ฅธ ์šฐํšŒ์ ์ธ ๋ฐฉ๋ฒ•์œผ๋กœ ๋ฐฐ์†ก์ƒํƒœ๋ฅผ ์•Œ ์ˆ˜ ์žˆ๋Š” ๋ฐฉ๋ฒ•์ด ์—†์„๊นŒ ํ•˜๊ณ  ์—ด์‹ฌํžˆ naver๋ฅผ ํƒ์ƒ‰ํ–ˆ์ง€๋งŒ ๊ฒฐ๊ตญ ์ด ๋ฐฉ๋ฒ•๋งŒ์ด ์œ ์ผํ•˜๊ฒŒ ๋ฐฐ์†ก์ƒํƒœ๋ฅผ ๋ณผ ์ˆ˜ ์žˆ์—ˆ๋‹ค.

playwright์—์„  popup๋„ ์ง€์›ํ•ด์ค„๊นŒ?

page.waitForEvent(โ€popupโ€) ์„ ํ†ตํ•ด ๊ฐ€๋Šฅํ•˜๋‹ค. Alt text

const popup = await popupPromise; ์ด ๊ตฌ๋ฌธ์„ ํ†ตํ•ด ์œ„์—์„œ ๋Œ€๊ธฐํ–ˆ๋˜ popup์ด๋ฒคํŠธ๊ฐ€ ์‹ค์ œ๋กœ ํ™”๋ฉด์— popup์ด ๋  ๋•Œ๊นŒ์ง€ ๊ธฐ๋‹ค๋ ธ๋‹ค๊ฐ€(await), popup ๋ณ€์ˆ˜์— ์ฐธ์กฐ๋ฅผ ์ €์žฅํ•œ๋‹ค.

์ด์ œ popup ์š”์†Œ ๋‚ด์˜ ํ•„์š”ํ•œ ์ •๋ณด๋“ค์„ ๊ฐ€์ ธ์˜ฌ ์ˆ˜ ์žˆ๋‹ค. Alt text

์Šฌ๋ž™๋ด‡ ์—ฐ๋™

initCrawler() ๋ฅผ ํ†ตํ•ด ํ•จ์ˆ˜๋ฅผ ์‹คํ–‰ ์‹œํ‚จ ํ›„ ๊ฒฐ๊ณผ๋“ค์„ ๊ฐ€๋…์„ฑ์žˆ๊ฒŒ ํฌ๋งคํŒ… ํ•œ ํ›„, slack api๋ฅผ ํ™œ์šฉํ•˜์—ฌ ๋ฐ์ดํ„ฐ๋ฅผ ์ „์†กํ•œ๋‹ค.

์—ฌ๋Ÿฌ๊ฐœ์˜ ์ž๋ฃŒ๋“ค์„ ๋ณด์•˜๋Š”๋ฐ, ์•„๋ž˜์˜ ์Šฌ๋ž™ ๊ณต์‹ ๋ฌธ์„œ ๋ณด๋‹ค ๋” ํ™•์‹คํ•œ๊ฑด ์—†๋‹ค.

Quickstart


Alt text ํƒ๋ฐฐํšŒ์‚ฌ๋ฅผ ํ†ตํ•œ ์ผ๋ฐ˜์ ์ธ ์ƒํ’ˆ์˜ ๋ฐฐ์†ก Alt text

์ƒ์ ์—์„œ ์ง์ ‘ ๋ฐฐ๋‹ฌํ•˜๋Š” ๊ฒฝ์šฐ์—” ์ฃผ๋ฌธ์ƒํƒœ์— ์ƒ์ ์—์„œ ๋‚จ๊ธด ๋ฉ”์‹œ์ง€๋ฅผ ๋„˜๊ฒจ ์ค˜์•ผ ํ•œ๋‹ค. lambda-slack


๊ฒฐ๊ณผ

๋น ๋ฅด๊ฒŒ ๋งŒ๋“ค์–ด ๋ณธ ๋ฐฐ์†ก ์ •๋ณด ํŠธ๋ž˜์ปค๋Š”, 5 ๋‹จ๊ณ„๋ฅผ ๊ฑธ์นœ ์š”์ฒญ ์ฒ˜๋ฆฌ์—์„œ ์ด๋ฏธ ์ฐฝ์— ๋„์–ด์ง„ ์Šฌ๋ž™ ์ฑ„๋„์—์„œ cmd+f๋ฅผ ํ†ตํ•ด ์ฃผ๋ฌธ ์ •๋ณด๋ฅผ ์ฐพ๋Š” ๊ณผ์ •์œผ๋กœ ๋๋‚ผ ์ˆ˜ ์žˆ๋‹ค.๋ฌด๋ ค 2~3๋‹จ๊ณ„๊ฐ€ ๊ฐ์ถ•๋˜๋Š” ํšจ๊ณผ๋ฅผ ๊ฐ€์ ธ์™”๋‹ค. ์•„์ง ๊ฐœ์„ ํ•  ์ ์ด ๋งŽ์ง€๋งŒ ํŒ€์— ๋„์›€์ด ๋œ๋‹ค๋Š” ๋ถ€๋ถ„์—์„œ ๋ฟŒ๋“ฏํ•œ ๊ฒฝํ—˜์„ ํ•  ์ˆ˜ ์žˆ์—ˆ๋‹ค. Alt text

์ฐธ๊ณ  ์ž๋ฃŒ

https://www.pluralsight.com/resources/blog/cloud/serverless-browser-automation-with-aws-lambda-and-puppeteer

https://docs.aws.amazon.com/ko_kr/lambda/latest/dg/gettingstarted-limits.html

Hi, I'm Hyunsu ๐Ÿ‘‹

Profile Image

์•ˆ๋…•ํ•˜์„ธ์š”. ํ”„๋ก ํŠธ์—”๋“œ ๊ฐœ๋ฐœ์ž ์ฃผํ˜„์ˆ˜์ž…๋‹ˆ๋‹ค.

๊ฐœ๋ฐœ์„ ํ†ตํ•ด ์‚ฌ์šฉ์ž๋“ค์—๊ฒŒ ํ’๋ถ€ํ•˜๊ณ  ๊ฐ€์น˜ ์žˆ๋Š” ๊ฒฝํ—˜์„ ์ œ๊ณตํ•˜๋Š” ์ผ์— ๋ฟŒ๋“ฏํ•จ์„ ๋Š๋‚๋‹ˆ๋‹ค.

์˜ต์‹œ๋””์–ธ(Obsidian)์—์„œ ํ˜„์žฌ ๋ธ”๋กœ๊ทธ๋กœ ํ•˜๋‚˜์”ฉ ๊ธ€์„ ์˜ฎ๊ธฐ๋Š” ๊ณผ์ •์— ์žˆ์–ด์š”. โ˜•๏ธ ๐Ÿ‘ฉโ€๐Ÿ’ป โ›ท

Github on ViewReach Me Out