Skip to content

Commit f959bcc

Browse files
abdurriqabdurriq
committed
Add tests
1 parent 16c0297 commit f959bcc

8 files changed

Lines changed: 273 additions & 3 deletions

File tree

‎src/artifacts-helper/README.md‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,10 +73,40 @@ pip install <package_name> --index-url https://pkgs.dev.azure.com/<org_name>/_pa
7373
When the feed URL is an Azure Artifacts feed pip will use the keyring helper to provide the credentials needed
7474
to download the package.
7575

76+
## Authentication Helper Wait Behavior
77+
78+
The shim scripts (e.g., `dotnet`, `npm`, `nuget`) now include a wait mechanism for the Azure DevOps authentication helper. When invoked, these scripts will:
79+
80+
1. Wait up to 3 minutes for the `ado-auth-helper` to become available
81+
2. Display progress indicators every 20 seconds while waiting
82+
3. Continue execution once authentication is successful
83+
4. Return an error (but not terminate dependent scripts) if the helper is not available after the timeout
84+
85+
This ensures that package restore operations can proceed even if there's a slight delay in the authentication helper installation, which can occur in some codespace initialization scenarios.
86+
87+
The scripts are designed to be sourced safely, meaning they won't terminate the calling shell if authentication fails - they will simply return an error code that can be handled by the calling script.
88+
7689
## OS Support
7790

7891
This feature is tested to work on Debian/Ubuntu and Mariner CBL 2.0
7992

93+
## Testing
94+
95+
To test this feature locally, you can use the devcontainer CLI:
96+
97+
```bash
98+
# Test all scenarios
99+
devcontainer features test -f artifacts-helper
100+
101+
# Test specific scenario
102+
devcontainer features test -f artifacts-helper --scenario test_auth_wait
103+
```
104+
105+
The test suite includes:
106+
- **test_auth_wait.sh**: Verifies that auth-ado.sh can be sourced without terminating the shell
107+
- **test_shim_integration.sh**: Tests that shim scripts properly handle missing authentication helper
108+
- **Python keyring tests**: Validates Python package installation with Azure Artifacts authentication
109+
80110
## Changing where functions are configured
81111

82112
By default, the functions are defined in `/etc/bash.bashrc` and `/etc/zsh/zshrc` if the container user is `root`, otherwise `~/.bashrc` and `~/.zshrc`.

‎src/artifacts-helper/install.sh‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,8 @@ mkdir -p "${SHIM_DIRECTORY}"
9292
# Install helper scripts in ${SHIM_DIRECTORY}
9393
cp "./scripts/auth-ado.sh" "${SHIM_DIRECTORY}"
9494
cp "./scripts/resolve-shim.sh" "${SHIM_DIRECTORY}"
95+
cp "./scripts/write-npm.sh" "${SHIM_DIRECTORY}"
96+
chmod +rx "${SHIM_DIRECTORY}/write-npm.sh"
9597

9698
# Install selected shim scripts in ${SHIM_DIRECTORY}
9799
for alias in "${ALIASES_ARR[@]}"; do

‎test/artifacts-helper/README.md‎

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
# Testing Azure Artifacts Helper
2+
3+
This document describes how to test the artifacts-helper feature, particularly the authentication wait behavior and shim script resilience.
4+
5+
## Test Scenarios
6+
7+
### 1. Authentication Wait Test (`test_auth_wait`)
8+
9+
**Purpose**: Verify that the auth-ado.sh script can be sourced without terminating the parent shell.
10+
11+
**What it tests**:
12+
- Shim scripts exist in `/usr/local/share/codespace-shims/`
13+
- `auth-ado.sh` can be sourced multiple times without crashing
14+
- Sourcing the script doesn't terminate the parent shell even on error
15+
16+
**Expected behavior**:
17+
- Scripts are executable and in the correct location
18+
- Sourcing auth-ado.sh returns control to the caller
19+
- Parent shell continues executing after sourcing fails
20+
21+
### 2. Shim Integration Test (`test_shim_integration`)
22+
23+
**Purpose**: Test that shim scripts properly handle missing authentication helper.
24+
25+
**What it tests**:
26+
- Shim scripts source auth-ado.sh correctly
27+
- Scripts handle timeout gracefully when ado-auth-helper is missing
28+
- Shim directory is in PATH
29+
- Scripts don't crash when authentication fails
30+
31+
**Expected behavior**:
32+
- Shims wait for up to 3 minutes for authentication
33+
- Timeout error is returned but doesn't crash the script
34+
- Calling scripts can continue or handle error appropriately
35+
36+
### 3. Python Keyring Tests
37+
38+
Multiple scenarios test Python integration:
39+
- `python38_and_keyring_debian`: Python 3.8 on Debian with keyring
40+
- `python38_and_keyring_ubuntu`: Python 3.8 on Ubuntu with keyring
41+
- `python312_and_keyring_debian`: Python 3.12 on Debian with keyring
42+
- `python_and_no_keyring`: Python without keyring helper
43+
44+
## Running Tests
45+
46+
### Run All Tests
47+
48+
```bash
49+
cd /path/to/codespace-features
50+
devcontainer features test -f artifacts-helper
51+
```
52+
53+
### Run Specific Test Scenario
54+
55+
```bash
56+
devcontainer features test -f artifacts-helper --scenario test_auth_wait
57+
devcontainer features test -f artifacts-helper --scenario test_shim_integration
58+
```
59+
60+
### Run Individual Test Script
61+
62+
If you want to test a specific script in an already-built container:
63+
64+
```bash
65+
# Inside a devcontainer with artifacts-helper installed
66+
bash /path/to/test/artifacts-helper/test_auth_wait.sh
67+
```
68+
69+
## Manual Testing
70+
71+
### Test Authentication Wait Behavior
72+
73+
1. Create a test devcontainer with artifacts-helper feature
74+
2. Remove or delay the ado-auth-helper installation
75+
3. Try to run a package manager command (e.g., `dotnet restore`)
76+
4. Observe that the script waits and shows progress
77+
5. Verify the script eventually times out with error but doesn't crash
78+
79+
```bash
80+
# Mock a scenario where ado-auth-helper is missing
81+
rm -f ~/ado-auth-helper
82+
83+
# Try to run dotnet - should wait and timeout gracefully
84+
timeout 10 dotnet --version
85+
echo "Exit code: $?" # Should be non-zero but script continues
86+
87+
# Verify we can still run commands
88+
echo "Shell is still active"
89+
```
90+
91+
### Test Shim Sourcing Behavior
92+
93+
```bash
94+
# Test that sourcing doesn't exit the shell
95+
bash -c '
96+
source /usr/local/share/codespace-shims/auth-ado.sh 2>/dev/null || echo "Returned with error"
97+
echo "Shell still running"
98+
'
99+
```
100+
101+
### Test with Actual Authentication
102+
103+
1. Set up a codespace with the ADO Codespaces Auth extension
104+
2. Configure an Azure Artifacts feed
105+
3. Wait for ado-auth-helper to be installed
106+
4. Run package restore commands
107+
5. Verify authentication succeeds
108+
109+
## What Changed in PR #85
110+
111+
The key changes improve resilience when the authentication helper isn't immediately available:
112+
113+
1. **Added wait loop**: Scripts now wait up to 3 minutes for ado-auth-helper
114+
2. **Removed `set -e`**: Prevents sourced script from terminating parent shell
115+
3. **Changed `exit` to `return`**: Allows error handling in calling scripts
116+
4. **Added progress indicators**: Shows wait progress every 20 seconds
117+
5. **Fixed PATH**: Uses hardcoded `/usr/local/share/codespace-shims` instead of variable
118+
119+
## Troubleshooting Test Failures
120+
121+
### "auth-ado.sh terminates shell"
122+
- Check that `set -e` is removed from auth-ado.sh
123+
- Verify `return 1` is used instead of `exit 1`
124+
125+
### "Shim scripts not found"
126+
- Verify PATH includes `/usr/local/share/codespace-shims`
127+
- Check that install.sh properly creates the shim scripts
128+
- Ensure containerEnv in devcontainer-feature.json is correct
129+
130+
### "Tests timeout"
131+
- Reduce MAX_WAIT in auth-ado.sh for faster testing
132+
- Use `timeout` command to limit test duration
133+
- Check that progress indicators are working
134+
135+
## CI/CD Integration
136+
137+
These tests are designed to work with the devcontainer features test framework and can be integrated into CI/CD pipelines:
138+
139+
```yaml
140+
- name: Test artifacts-helper feature
141+
run: |
142+
devcontainer features test -f artifacts-helper
143+
```

‎test/artifacts-helper/run-tests.sh‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
#!/bin/bash
2+
# Quick test runner for artifacts-helper feature
3+
# Usage: ./run-tests.sh [scenario-name]
4+
5+
set -e
6+
7+
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
8+
FEATURE_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
9+
10+
echo "==================================="
11+
echo "Testing artifacts-helper feature"
12+
echo "==================================="
13+
echo
14+
15+
if [ -n "$1" ]; then
16+
echo "Running scenario: $1"
17+
devcontainer features test -f artifacts-helper --scenario "$1"
18+
else
19+
echo "Running all scenarios..."
20+
devcontainer features test -f artifacts-helper
21+
fi
22+
23+
echo
24+
echo "==================================="
25+
echo "Tests completed!"
26+
echo "==================================="

‎test/artifacts-helper/scenarios.json‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,5 +42,21 @@
4242
"python": false
4343
}
4444
}
45+
},
46+
"test_auth_wait": {
47+
"image": "mcr.microsoft.com/devcontainers/base:debian",
48+
"features": {
49+
"artifacts-helper": {}
50+
}
51+
},
52+
"test_shim_integration": {
53+
"image": "mcr.microsoft.com/devcontainers/base:ubuntu",
54+
"features": {
55+
"artifacts-helper": {
56+
"dotnetAlias": true,
57+
"npmAlias": true,
58+
"nugetAlias": true
59+
}
60+
}
4561
}
4662
}

‎test/artifacts-helper/test.sh‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@ set -e
66
source dev-container-features-test-lib
77

88
# Feature-specific tests
9-
check "dotnet" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-dotnet.sh)
10-
check "nuget" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-nuget.sh)
11-
check "write-npm" /usr/local/bin/write-npm.sh pkgs.dev.azure.com && grep "pkgs.dev.azure.com" <(cat ~/.npmrc)
9+
check "dotnet" grep "pkgs.dev.azure.com" <(cat /usr/local/share/codespace-shims/dotnet)
10+
check "nuget" grep "pkgs.dev.azure.com" <(cat /usr/local/share/codespace-shims/nuget)
11+
check "write-npm" /usr/local/share/codespace-shims/write-npm.sh pkgs.dev.azure.com && grep "pkgs.dev.azure.com" <(cat ~/.npmrc)
1212

1313

1414
# Report results

‎test/artifacts-helper/test_auth_wait.sh‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
#!/usr/bin/env bash
2+
3+
# Import test library bundled with the devcontainer CLI
4+
source dev-container-features-test-lib || exit 1
5+
6+
# Test that the shim scripts exist and can be sourced
7+
check "dotnet shim exists" test -f /usr/local/share/codespace-shims/dotnet
8+
check "npm shim exists" test -f /usr/local/share/codespace-shims/npm
9+
check "nuget shim exists" test -f /usr/local/share/codespace-shims/nuget
10+
11+
# Test that auth-ado.sh can be sourced without exiting the shell
12+
check "auth-ado.sh can be sourced" bash -c 'source /usr/local/share/codespace-shims/auth-ado.sh 2>/dev/null || true; echo "still running"'
13+
14+
# Test that sourcing auth-ado.sh doesn't terminate the parent shell
15+
check "sourcing auth-ado.sh preserves shell" bash -c '
16+
source /usr/local/share/codespace-shims/auth-ado.sh 2>/dev/null || true
17+
echo "completed"
18+
' | grep -q "completed"
19+
20+
# Test that the shim scripts can be executed
21+
check "dotnet shim is executable" test -x /usr/local/share/codespace-shims/dotnet
22+
check "npm shim is executable" test -x /usr/local/share/codespace-shims/npm
23+
24+
# Report results
25+
reportResults

‎test/artifacts-helper/test_shim_integration.sh‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
#!/usr/bin/env bash
2+
3+
# Import test library bundled with the devcontainer CLI
4+
source dev-container-features-test-lib || exit 1
5+
6+
# Test that shims properly handle the case when ado-auth-helper is not available
7+
# The shim should wait and eventually timeout, but not crash the script
8+
9+
# Test dotnet shim can be invoked (will timeout waiting for auth but shouldn't crash)
10+
check "dotnet shim handles missing auth helper" bash -c '
11+
# Mock a quick timeout scenario
12+
export HOME=$(mktemp -d)
13+
timeout 5 /usr/local/share/codespace-shims/dotnet --version 2>&1 || exit_code=$?
14+
# Exit code 124 means timeout killed it, which is expected
15+
# Exit code 1 means it returned error but script continued
16+
# Exit code 0 means it succeeded (if auth helper was present)
17+
[[ $exit_code -eq 124 || $exit_code -eq 1 || $exit_code -eq 0 ]]
18+
'
19+
20+
# Test that the shim scripts properly source auth-ado.sh
21+
check "dotnet shim sources auth-ado.sh" grep -q "source.*auth-ado.sh" /usr/local/share/codespace-shims/dotnet
22+
check "npm shim sources auth-ado.sh" grep -q "source.*auth-ado.sh" /usr/local/share/codespace-shims/npm
23+
24+
# Verify the shim directory is in PATH
25+
check "shim directory in PATH" bash -c '[[ ":$PATH:" == *":/usr/local/share/codespace-shims:"* ]]'
26+
27+
# Report results
28+
reportResults

0 commit comments

Comments
 (0)