Repository navigation
fix(ai-analyzer): pin LOKI_URL to log-server Tailscale IP - #80
Merged
Merged
Conversation
검증 중 ai-analyzer pod 의 Loki 조회가 매번 `[Errno -2] Name or service not known` 로 실패하는 것을 확인. pod 의 resolv.conf 가 cluster CoreDNS (10.43.0.10) → Tailscale DNS (100.100.100.100) 순서인데, CoreDNS 가 `tail07adfd.ts.net` 에 대해 NXDOMAIN 을 반환해서 glibc resolver 가 두 번째 nameserver 로 fallback 하지 않음. dnsConfig.nameservers 가 박혀 있어도 우선순위가 안 먹는 구조적 한계. #76 (otel-gateway export endpoint) 와 같은 패턴으로 Tailscale IP (observ-log-server = 100.68.2.91) 를 직접 박아 우회. README 환경변수 표와 values.yaml 주석에 이유 명시. PR #78 (logql_context_query annotation) 의 실측 검증 prerequisite. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ai-analyzer pod의 cross-cluster Loki 조회 실패를 우회.
apps/ai/ai-analyzer/chart/values.yaml의LOKI_URL을 MagicDNS FQDN(observ-log-server.tail07adfd.ts.net) 대신 Tailscale IP(100.68.2.91)로 변경. #76 (otel-gateway → Tailscale IP) 패턴과 동일.Why
PR #78의
logql_context_queryannotation 흐름을 실측 검증하던 중, 매 webhook 호출에서loki query failed: [Errno -2] Name or service not known가 발생함을 확인.진단 결과:
socket.gethostbyname → [Errno -2]100.68.2.91로는 도달 OKTCP OK,HTTP /ready 200nameserver 10.43.0.10/nameserver 100.100.100.100tail07adfd.ts.net박혀 있음근본 원인: CoreDNS가
tail07adfd.ts.net에 대해 forward zone 없이 NXDOMAIN을 반환하고, glibc resolver는 첫 nameserver가 NXDOMAIN을 주면 두 번째 nameserver로 fallback하지 않음.dnsConfig.nameservers로 100.100.100.100을 박아둬도 순서상 우선순위가 안 먹는 구조적 한계.kubectl set env임시 override도 ArgoCDselfHeal에 즉시 원복되어 검증 불가.#76(otel-gateway export endpoint → Tailscale IP) 가 정확히 같은 함정을 해결한 패턴이라 같은 길로 답습. README와 values.yaml 주석에 사유 명시해 다음 사람이 다시 풀게 하지 않음.Validation
helm template apps/ai/ai-analyzer/chart(변경 후 정상 렌더)LOKI_URL환경변수가http://100.68.2.91:31100로 박히는지loki_lines > 0loki_lines > 0loki_lines == 0Notes
tail07adfd.ts.netforward zone을 추가하는 것이지만, 인프라 변경 범위가 커서 이번 PR 범위 밖.Related: #76, PR #78
🤖 Generated with Claude Code