[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"summary-6e0d96342ee641a1-the-art-science-of-benchmarking-ai-agents-summary":3,"summaries-facets-categories":173,"summary-related-6e0d96342ee641a1-the-art-science-of-benchmarking-ai-agents-summary":7077},{"id":4,"title":5,"ai":6,"body":13,"categories":116,"created_at":118,"date_modified":118,"description":110,"extension":119,"faq":118,"featured":120,"kicker_label":118,"meta":121,"navigation":152,"path":153,"published_at":154,"question":118,"scraped_at":155,"seo":156,"sitemap":157,"source_id":158,"source_name":159,"source_type":160,"source_url":161,"stem":162,"tags":163,"thumbnail_url":168,"tldr":169,"tweet":170,"unknown_tags":171,"__hash__":172},"summaries\u002Fsummaries\u002F6e0d96342ee641a1-the-art-science-of-benchmarking-ai-agents-summary.md","The Art & Science of Benchmarking AI Agents",{"provider":7,"model":8,"input_tokens":9,"output_tokens":10,"processing_time_ms":11,"cost_usd":12},"openrouter","google\u002Fgemini-3.1-flash-lite",8202,995,5199,0.003543,{"type":14,"value":15,"toc":109},"minimark",[16,21,25,54,58,61,81,85,88],[17,18,20],"h2",{"id":19},"the-science-of-effective-benchmarks","The Science of Effective Benchmarks",[22,23,24],"p",{},"To build a benchmark that actually shapes the field, builders must move beyond simple accuracy metrics and focus on four empirical pillars:",[26,27,28,36,42,48],"ul",{},[29,30,31,35],"li",{},[32,33,34],"strong",{},"Individual Task Quality:"," Tasks must be rigorously validated, well-posed, and tractable for experts. The author highlights GPQA for its adversarial quality control, where multi-expert protocols and incentive mechanisms ensure tasks are not just difficult, but correctly structured.",[29,37,38,41],{},[32,39,40],{},"Distributional Diversity:"," A benchmark is only as good as its coverage. Builders should define a clear taxonomy of the domain and intentionally distribute tasks across it, including rare but critical failure modes. MMLU is cited as a gold standard for its intentional taxonomy across 57 domains.",[29,43,44,47],{},[32,45,46],{},"Model Headroom:"," Benchmarks must remain unsaturated to effectively separate frontier models. The ARC Prize is highlighted as a model for this, as it consistently exposes the gap between human reasoning and model capabilities, reliably predicting leaps in model performance.",[29,49,50,53],{},[32,51,52],{},"Robust Eval Methodology:"," Evaluation must capture real-world constraints beyond simple completion. The author points to ToW-Bench, which evaluates agents not just on task success, but on adherence to policy constraints (e.g., failing a flight booking if it violates class rules).",[17,55,57],{"id":56},"the-art-of-shaping-the-frontier","The Art of Shaping the Frontier",[22,59,60],{},"Beyond empirical rigor, the most influential benchmarks act as strategic bets that guide the research community:",[26,62,63,69,75],{},[29,64,65,68],{},[32,66,67],{},"Thesis-Driven Design:"," Great benchmarks represent a bet on where the field is going. Terminal Bench, for example, was a bet that the CLI would become a primary interface for agents—a bet that has since been validated by the industry.",[29,70,71,74],{},[32,72,73],{},"Roadmap Generation:"," A successful benchmark spawns a family of research. SWE-bench is praised for its simplicity and its ability to inspire a new generation of coding-agent benchmarks (e.g., light, verified, multimodal versions), creating a clear path for future innovation.",[29,76,77,80],{},[32,78,79],{},"Researcher UX:"," This is a severely underrated factor. If a benchmark is difficult to run, extend, or use for RL\u002Ffine-tuning, it will not be adopted. Building standardized harnesses (like HELM or Harbor) is essential for ensuring that the community can easily hill-climb against the benchmark.",[17,82,84],{"id":83},"the-next-generation-of-benchmarks","The Next Generation of Benchmarks",[22,86,87],{},"To push the frontier further, the author proposes three new axes for future benchmark development:",[89,90,91,97,103],"ol",{},[29,92,93,96],{},[32,94,95],{},"Environment Complexity:"," Moving beyond isolated tasks to represent real-world \"messiness,\" such as organizational policies, flaky toolchains, and multi-modal context.",[29,98,99,102],{},[32,100,101],{},"Autonomy Horizon:"," Measuring reliability over long-term, multi-week interactions where context shifts, requirements change, and state management becomes the primary challenge.",[29,104,105,108],{},[32,106,107],{},"Output Complexity:"," Expanding evaluation beyond text to include nuanced reward signals and diverse artifacts that reflect actual professional workflows.",{"title":110,"searchDepth":111,"depth":111,"links":112},"",2,[113,114,115],{"id":19,"depth":111,"text":20},{"id":56,"depth":111,"text":57},{"id":83,"depth":111,"text":84},[117],"AI & LLMs",null,"md",false,{"content_references":122,"triage":147},[123,128,131,135,138,141,144],{"type":124,"title":125,"url":126,"context":127},"tool","GPQA","https:\u002F\u002Fgithub.com\u002Fidavidrein\u002Fgpqa","recommended",{"type":124,"title":129,"url":130,"context":127},"ARC Prize","https:\u002F\u002Farcprize.org\u002F",{"type":124,"title":132,"url":133,"context":134},"MMLU","https:\u002F\u002Fgithub.com\u002Fhendrycks\u002Ftest","mentioned",{"type":124,"title":136,"url":137,"context":127},"ToW-Bench","https:\u002F\u002Fgithub.com\u002Ftow-bench\u002Ftow-bench",{"type":124,"title":139,"url":140,"context":127},"Terminal Bench","https:\u002F\u002Fterminal-bench.github.io\u002F",{"type":124,"title":142,"url":143,"context":127},"SWE-bench","https:\u002F\u002Fwww.swebench.com\u002F",{"type":124,"title":145,"url":146,"context":127},"HELM","https:\u002F\u002Fcrfm.stanford.edu\u002Fhelm\u002Flatest\u002F",{"relevance":148,"novelty":148,"quality":148,"actionability":149,"composite":150,"reasoning":151},4,3,3.8,"Category: AI & LLMs. The article discusses effective benchmarking for AI agents, which is crucial for builders looking to evaluate and improve AI capabilities. It provides specific empirical pillars for creating benchmarks, addressing a pain point for developers needing practical evaluation methods.",true,"\u002Fsummaries\u002F6e0d96342ee641a1-the-art-science-of-benchmarking-ai-agents-summary","2026-06-04 16:00:06","2026-06-06 16:08:56",{"title":5,"description":110},{"loc":153},"6e0d96342ee641a1","AI Engineer","video","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=iNkFlCiij0U","summaries\u002F6e0d96342ee641a1-the-art-science-of-benchmarking-ai-agents-summary",[164,165,166,167],"agents","research","ai-llms","evaluation","https:\u002F\u002Fi.ytimg.com\u002Fvi\u002FiNkFlCiij0U\u002Fhqdefault.jpg","Effective AI benchmarks are not just snapshots of current performance; they are strategic tools that define future capabilities, require rigorous task quality, and prioritize researcher UX to drive field-wide progress.","This talk outlines a framework for designing AI benchmarks, drawing on lessons from reviewing over 120 grant applications at [Snorkel AI](https:\u002F\u002Fsnorkel.ai). The speaker argues that effective benchmarks require both scientific rigor—focusing on task quality, distributional diversity, and model headroom—and an artistic \"thesis\" that anticipates future capabilities rather than just measuring current ones.",[166,167],"ZEgYR3dr9isWQwq1xAh-EB-r4KoE5sMBD19yp8pCfJw",[174,176,179,181,184,186,189,192,194,196,198,200,203,205,207,209,211,214,216,218,220,222,225,228,230,232,234,236,238,240,242,244,246,248,250,252,254,256,258,260,262,264,266,268,270,272,274,277,279,281,283,285,287,289,291,293,295,297,299,301,303,306,308,310,312,314,316,318,320,322,324,326,328,330,332,334,336,338,340,342,344,347,349,351,353,355,357,359,361,363,365,367,369,371,373,376,378,380,382,384,386,388,390,392,394,396,398,400,402,404,406,408,410,412,414,416,418,420,422,424,426,428,430,432,434,436,438,441,443,445,447,449,451,453,455,457,459,461,464,466,468,470,472,474,476,478,480,482,484,486,488,490,492,494,497,499,501,503,505,507,509,511,513,515,517,520,522,524,526,528,530,532,534,536,538,540,542,544,546,548,550,552,554,556,558,560,562,564,566,568,570,572,574,576,579,581,583,586,588,590,592,594,596,598,600,602,604,606,608,610,612,614,616,618,620,622,625,627,629,631,633,635,637,639,641,643,645,647,650,652,654,656,658,660,662,664,666,668,670,672,674,676,678,680,682,684,686,688,690,692,694,696,698,700,702,704,706,708,710,712,714,716,718,720,722,724,726,728,730,732,734,736,738,740,742,744,746,748,750,752,754,756,758,760,762,764,766,768,770,772,774,776,778,780,782,784,786,788,790,792,794,796,798,800,802,804,806,808,810,812,814,816,818,820,822,824,826,828,830,832,834,836,838,840,842,844,846,848,850,852,854,856,858,860,862,864,866,868,870,872,874,876,878,880,882,884,886,888,890,892,894,896,898,900,902,904,906,908,910,912,914,916,918,920,922,924,926,928,930,932,934,937,939,941,943,945,948,950,952,954,956,958,960,962,964,966,968,970,972,975,977,979,981,983,985,987,989,991,993,995,997,999,1001,1003,1005,1007,1009,1011,1013,1015,1017,1019,1021,1023,1025,1027,1029,1031,1033,1035,1037,1039,1041,1043,1045,1047,1049,1051,1053,1055,1057,1059,1061,1063,1065,1067,1069,1071,1073,1075,1077,1079,1081,1083,1085,1087,1089,1091,1093,1095,1097,1099,1101,1103,1105,1107,1109,1111,1113,1115,1117,1119,1121,1123,1125,1127,1129,1131,1133,1135,1137,1139,1141,1143,1145,1147,1149,1151,1153,1155,1157,1159,1161,1163,1165,1167,1169,1171,1173,1175,1177,1179,1181,1183,1185,1187,1189,1191,1193,1195,1197,1199,1201,1203,1205,1207,1209,1211,1213,1215,1217,1219,1221,1223,1225,1227,1229,1231,1233,1235,1237,1239,1241,1243,1245,1247,1249,1251,1253,1255,1257,1259,1261,1263,1265,1267,1269,1271,1273,1275,1277,1279,1281,1283,1285,1287,1289,1291,1293,1295,1297,1300,1302,1304,1306,1308,1310,1312,1314,1316,1318,1320,1322,1324,1326,1328,1330,1332,1334,1336,1338,1340,1342,1344,1346,1348,1350,1352,1354,1356,1358,1360,1362,1364,1366,1368,1370,1372,1374,1376,1378,1380,1382,1384,1386,1388,1390,1392,1394,1396,1398,1400,1402,1404,1406,1408,1410,1412,1414,1416,1418,1420,1422,1424,1426,1428,1430,1432,1434,1436,1438,1440,1442,1444,1446,1448,1450,1452,1454,1456,1458,1460,1462,1464,1466,1468,1470,1472,1474,1476,1478,1480,1482,1484,1486,1488,1490,1492,1494,1496,1498,1501,1503,1505,1507,1509,1511,1513,1515,1517,1519,1521,1523,1525,1527,1529,1531,1533,1535,1537,1539,1541,1543,1545,1547,1549,1551,1553,1555,1557,1559,1561,1563,1565,1567,1569,1571,1573,1575,1577,1579,1581,1583,1585,1587,1589,1591,1593,1595,1597,1599,1601,1603,1605,1607,1609,1611,1613,1615,1617,1619,1621,1623,1625,1627,1629,1631,1633,1635,1637,1640,1642,1644,1646,1648,1650,1652,1654,1656,1658,1660,1662,1664,1666,1668,1670,1672,1674,1676,1678,1680,1682,1684,1686,1688,1690,1692,1694,1696,1698,1700,1702,1704,1706,1708,1710,1712,1714,1716,1718,1720,1722,1724,1726,1728,1730,1732,1734,1736,1738,1740,1742,1744,1746,1748,1750,1752,1754,1756,1758,1760,1762,1764,1766,1768,1770,1772,1774,1776,1778,1780,1782,1784,1786,1788,1790,1792,1794,1796,1799,1801,1803,1805,1807,1809,1811,1813,1815,1817,1819,1821,1823,1825,1827,1829,1831,1833,1835,1837,1839,1841,1843,1845,1847,1849,1851,1853,1855,1857,1859,1862,1864,1866,1868,1870,1872,1874,1876,1878,1880,1882,1884,1886,1888,1890,1892,1894,1896,1898,1900,1902,1904,1906,1908,1910,1912,1914,1916,1918,1920,1922,1924,1926,1928,1930,1932,1934,1936,1938,1940,1942,1944,1946,1948,1950,1952,1954,1956,1958,1960,1962,1964,1966,1968,1970,1972,1974,1976,1978,1980,1982,1984,1986,1988,1990,1992,1994,1996,1998,2000,2002,2004,2006,2008,2010,2012,2014,2016,2018,2020,2022,2024,2026,2028,2030,2032,2034,2036,2038,2040,2042,2044,2046,2048,2050,2052,2054,2056,2058,2060,2062,2064,2066,2068,2070,2072,2074,2076,2078,2080,2082,2084,2086,2088,2090,2092,2094,2096,2098,2100,2102,2104,2106,2108,2110,2112,2114,2116,2118,2120,2122,2124,2126,2128,2130,2132,2134,2136,2138,2140,2142,2144,2146,2148,2150,2152,2154,2156,2158,2160,2162,2164,2166,2168,2170,2172,2174,2176,2178,2180,2182,2184,2186,2188,2190,2192,2194,2196,2198,2200,2202,2204,2206,2208,2210,2212,2214,2216,2218,2220,2222,2224,2226,2228,2230,2232,2234,2236,2238,2240,2242,2244,2246,2248,2250,2252,2254,2256,2258,2260,2262,2264,2266,2268,2270,2272,2274,2276,2278,2280,2282,2284,2286,2288,2290,2292,2294,2296,2298,2300,2303,2305,2307,2309,2311,2313,2315,2317,2319,2321,2323,2325,2327,2329,2331,2333,2335,2337,2339,2341,2343,2345,2347,2349,2351,2353,2355,2357,2359,2361,2363,2365,2367,2369,2371,2373,2375,2377,2379,2381,2383,2385,2387,2389,2391,2393,2396,2398,2400,2402,2404,2406,2408,2410,2412,2414,2416,2418,2420,2422,2424,2426,2428,2430,2432,2434,2436,2438,2440,2442,2444,2446,2448,2450,2452,2454,2456,2458,2460,2462,2464,2466,2468,2470,2472,2474,2476,2478,2480,2482,2484,2486,2488,2490,2492,2494,2496,2498,2500,2502,2504,2506,2508,2510,2512,2515,2517,2519,2521,2523,2525,2527,2529,2531,2533,2535,2537,2539,2541,2543,2545,2547,2549,2551,2553,2555,2558,2560,2562,2564,2566,2568,2570,2572,2574,2576,2578,2580,2582,2584,2586,2588,2590,2592,2594,2596,2598,2600,2602,2604,2606,2608,2610,2612,2614,2616,2618,2620,2622,2624,2626,2628,2630,2632,2634,2636,2638,2640,2642,2644,2646,2648,2650,2652,2654,2656,2658,2660,2662,2664,2666,2668,2670,2672,2674,2676,2678,2680,2682,2684,2686,2688,2690,2692,2694,2696,2698,2700,2702,2704,2706,2708,2710,2712,2714,2716,2718,2720,2722,2724,2726,2728,2730,2732,2734,2736,2738,2740,2742,2744,2746,2748,2750,2752,2754,2756,2758,2760,2762,2764,2766,2768,2770,2772,2774,2776,2778,2780,2782,2784,2786,2788,2790,2792,2794,2796,2798,2800,2802,2804,2806,2808,2810,2812,2814,2816,2818,2820,2822,2824,2826,2828,2830,2832,2834,2836,2838,2840,2842,2844,2846,2848,2850,2852,2854,2856,2858,2860,2862,2864,2866,2868,2870,2872,2874,2876,2878,2880,2882,2884,2886,2888,2890,2892,2894,2896,2898,2900,2902,2904,2906,2908,2910,2912,2914,2916,2918,2920,2922,2924,2926,2928,2930,2932,2934,2936,2938,2940,2942,2944,2946,2948,2950,2952,2954,2956,2958,2960,2962,2964,2966,2968,2970,2972,2974,2976,2978,2980,2982,2984,2986,2988,2990,2992,2994,2996,2998,3000,3002,3004,3006,3008,3010,3012,3014,3016,3018,3020,3022,3024,3026,3028,3030,3032,3034,3036,3038,3040,3042,3044,3046,3048,3050,3052,3054,3056,3058,3060,3062,3064,3066,3068,3070,3072,3074,3076,3078,3080,3082,3084,3086,3088,3090,3092,3094,3096,3098,3100,3102,3104,3106,3108,3110,3112,3114,3116,3118,3120,3122,3124,3126,3128,3130,3132,3134,3136,3138,3140,3142,3144,3146,3148,3150,3152,3154,3156,3158,3160,3162,3164,3166,3168,3170,3172,3174,3176,3178,3180,3182,3184,3186,3188,3190,3192,3194,3196,3199,3201,3203,3205,3207,3209,3211,3213,3215,3217,3219,3221,3223,3225,3227,3229,3231,3233,3235,3237,3239,3241,3243,3245,3247,3249,3251,3253,3255,3257,3259,3261,3263,3265,3267,3269,3271,3273,3275,3277,3279,3281,3283,3285,3287,3289,3291,3293,3295,3297,3299,3301,3303,3305,3307,3309,3311,3313,3315,3317,3319,3321,3323,3326,3328,3330,3332,3334,3336,3338,3340,3342,3344,3346,3348,3350,3352,3354,3356,3358,3360,3362,3364,3366,3368,3370,3372,3374,3376,3378,3380,3382,3384,3386,3388,3390,3392,3394,3396,3398,3400,3402,3404,3406,3408,3410,3412,3414,3416,3418,3420,3422,3424,3426,3428,3430,3432,3434,3436,3438,3440,3442,3444,3446,3448,3450,3452,3454,3456,3458,3460,3462,3464,3466,3468,3470,3472,3474,3476,3478,3480,3482,3484,3486,3488,3490,3492,3494,3496,3498,3500,3502,3504,3506,3508,3510,3512,3514,3516,3518,3520,3522,3524,3526,3528,3530,3532,3534,3536,3538,3540,3542,3544,3546,3548,3550,3552,3554,3556,3558,3560,3562,3564,3566,3568,3570,3572,3574,3576,3578,3580,3582,3584,3586,3588,3590,3592,3594,3596,3598,3600,3602,3604,3606,3608,3610,3612,3614,3616,3618,3620,3622,3624,3626,3628,3630,3632,3634,3636,3638,3640,3642,3644,3646,3648,3650,3652,3654,3656,3658,3660,3662,3664,3666,3668,3670,3672,3674,3676,3678,3680,3682,3684,3686,3688,3690,3692,3694,3696,3698,3700,3702,3704,3706,3708,3710,3712,3714,3716,3718,3720,3722,3724,3726,3728,3730,3732,3734,3736,3738,3740,3742,3744,3746,3748,3750,3752,3754,3756,3758,3760,3762,3764,3766,3768,3770,3772,3774,3776,3778,3780,3782,3784,3786,3788,3790,3792,3794,3796,3798,3800,3802,3804,3806,3808,3810,3812,3814,3816,3818,3820,3822,3824,3826,3828,3830,3832,3834,3836,3838,3840,3842,3844,3846,3848,3850,3852,3854,3856,3858,3860,3862,3864,3866,3868,3870,3872,3874,3876,3878,3880,3882,3884,3886,3888,3890,3892,3894,3896,3898,3900,3902,3904,3906,3908,3910,3912,3914,3916,3918,3920,3922,3924,3926,3928,3930,3932,3934,3936,3938,3940,3942,3944,3946,3948,3950,3952,3954,3956,3958,3960,3962,3964,3966,3968,3970,3972,3974,3976,3978,3980,3982,3984,3986,3988,3990,3992,3994,3996,3998,4000,4002,4004,4006,4008,4010,4012,4014,4016,4018,4020,4022,4024,4026,4028,4030,4032,4034,4036,4038,4040,4042,4044,4046,4048,4050,4052,4054,4056,4058,4060,4062,4064,4066,4068,4070,4072,4074,4076,4078,4080,4082,4084,4086,4088,4090,4092,4094,4096,4098,4100,4102,4104,4106,4108,4110,4112,4114,4116,4118,4120,4122,4124,4126,4128,4130,4132,4134,4136,4138,4140,4142,4144,4146,4148,4150,4152,4154,4156,4158,4160,4162,4164,4166,4168,4170,4172,4174,4176,4178,4180,4182,4184,4186,4188,4190,4192,4194,4196,4198,4200,4202,4204,4206,4208,4210,4212,4214,4216,4218,4220,4222,4224,4226,4228,4230,4232,4234,4236,4238,4240,4242,4244,4246,4248,4250,4252,4254,4256,4258,4260,4262,4264,4266,4268,4270,4272,4274,4276,4278,4280,4282,4284,4286,4288,4290,4292,4294,4296,4298,4300,4302,4304,4306,4308,4310,4312,4314,4316,4318,4320,4322,4324,4326,4328,4330,4332,4334,4336,4338,4340,4342,4344,4346,4348,4350,4352,4354,4356,4358,4360,4362,4364,4366,4368,4370,4372,4374,4376,4378,4380,4382,4384,4386,4388,4390,4392,4394,4396,4398,4400,4402,4404,4406,4408,4410,4412,4414,4416,4418,4420,4422,4424,4426,4428,4430,4432,4434,4436,4438,4440,4442,4444,4446,4448,4450,4452,4454,4456,4458,4460,4462,4464,4466,4468,4470,4472,4474,4476,4478,4480,4482,4484,4486,4488,4490,4492,4494,4496,4498,4500,4502,4504,4506,4508,4510,4512,4514,4516,4518,4520,4522,4524,4526,4528,4530,4532,4534,4536,4538,4540,4542,4544,4546,4548,4550,4552,4554,4556,4558,4560,4562,4564,4566,4568,4570,4572,4574,4576,4578,4580,4582,4584,4586,4588,4590,4592,4594,4596,4598,4600,4602,4604,4606,4608,4611,4613,4615,4617,4619,4621,4623,4625,4627,4629,4631,4633,4635,4637,4639,4641,4643,4645,4647,4649,4651,4653,4655,4657,4659,4661,4663,4665,4667,4669,4671,4673,4675,4677,4679,4681,4683,4685,4687,4689,4691,4693,4695,4697,4699,4701,4703,4705,4707,4709,4711,4713,4715,4717,4719,4721,4723,4725,4727,4729,4731,4733,4735,4737,4739,4741,4743,4745,4747,4749,4751,4753,4755,4757,4759,4761,4763,4765,4767,4769,4771,4773,4775,4777,4779,4781,4783,4785,4787,4789,4791,4793,4795,4797,4799,4801,4803,4805,4807,4809,4811,4813,4815,4817,4819,4821,4823,4825,4827,4829,4831,4833,4835,4837,4839,4841,4843,4845,4847,4849,4851,4853,4855,4857,4859,4861,4863,4865,4867,4869,4871,4873,4875,4877,4879,4881,4883,4885,4887,4889,4891,4893,4895,4897,4899,4901,4903,4905,4907,4909,4911,4913,4915,4917,4919,4921,4923,4925,4927,4929,4931,4933,4935,4937,4939,4941,4943,4945,4947,4949,4951,4953,4955,4957,4959,4961,4963,4965,4967,4969,4971,4973,4975,4977,4979,4981,4983,4985,4987,4989,4991,4993,4995,4997,4999,5001,5003,5005,5007,5009,5011,5013,5015,5017,5019,5021,5023,5025,5027,5029,5031,5033,5035,5037,5039,5041,5043,5045,5047,5049,5051,5053,5055,5057,5059,5061,5063,5065,5067,5069,5071,5073,5075,5077,5079,5081,5083,5085,5087,5089,5091,5093,5095,5097,5099,5101,5103,5105,5107,5109,5111,5113,5115,5117,5119,5121,5123,5125,5127,5129,5131,5133,5135,5137,5139,5141,5143,5145,5147,5149,5151,5153,5155,5157,5159,5161,5163,5165,5167,5169,5171,5173,5175,5177,5179,5181,5183,5185,5187,5189,5191,5193,5195,5197,5199,5201,5203,5205,5207,5209,5211,5213,5215,5217,5219,5221,5223,5225,5227,5229,5231,5233,5235,5237,5239,5241,5243,5245,5247,5249,5251,5253,5255,5257,5259,5261,5263,5265,5267,5269,5271,5273,5275,5277,5279,5281,5283,5285,5287,5289,5291,5293,5295,5297,5299,5301,5303,5305,5307,5309,5311,5313,5315,5317,5319,5321,5323,5325,5327,5329,5331,5333,5335,5337,5339,5341,5343,5345,5347,5349,5351,5353,5355,5357,5359,5361,5363,5365,5367,5369,5371,5373,5375,5377,5379,5381,5383,5385,5387,5389,5391,5393,5395,5397,5399,5401,5403,5405,5407,5409,5411,5413,5415,5417,5419,5421,5423,5425,5427,5429,5431,5433,5435,5437,5439,5441,5443,5445,5447,5449,5451,5453,5455,5457,5459,5461,5463,5465,5468,5470,5472,5474,5476,5478,5480,5482,5484,5486,5488,5490,5492,5494,5496,5498,5500,5502,5504,5506,5508,5510,5512,5514,5516,5518,5520,5522,5524,5526,5528,5530,5532,5534,5536,5538,5540,5542,5544,5546,5548,5550,5552,5554,5556,5558,5560,5562,5564,5566,5568,5570,5572,5574,5576,5578,5580,5582,5584,5586,5588,5590,5592,5594,5596,5598,5600,5602,5604,5606,5608,5610,5612,5614,5616,5618,5620,5622,5624,5626,5628,5630,5632,5634,5636,5638,5640,5642,5644,5646,5648,5650,5652,5654,5656,5658,5660,5662,5664,5666,5668,5671,5673,5675,5677,5679,5681,5683,5685,5687,5689,5691,5693,5695,5697,5699,5701,5703,5705,5707,5709,5711,5713,5715,5717,5719,5721,5723,5725,5727,5729,5731,5733,5735,5737,5739,5741,5743,5745,5747,5749,5751,5753,5755,5757,5759,5761,5763,5765,5767,5769,5771,5773,5775,5777,5779,5781,5783,5785,5787,5789,5791,5793,5795,5797,5799,5801,5803,5805,5807,5809,5811,5813,5815,5817,5819,5821,5823,5825,5827,5829,5831,5833,5835,5837,5839,5841,5843,5845,5847,5849,5851,5853,5855,5857,5859,5861,5863,5865,5867,5869,5871,5873,5875,5877,5879,5881,5883,5885,5887,5889,5891,5893,5895,5897,5899,5901,5903,5905,5907,5909,5911,5913,5915,5917,5919,5921,5923,5925,5927,5929,5931,5933,5935,5937,5939,5941,5943,5945,5947,5949,5951,5953,5955,5957,5959,5961,5963,5965,5967,5969,5971,5973,5975,5977,5979,5981,5983,5985,5987,5989,5991,5993,5995,5997,5999,6001,6003,6005,6007,6009,6011,6013,6015,6017,6019,6021,6023,6025,6027,6029,6031,6033,6035,6037,6039,6041,6043,6045,6047,6049,6051,6053,6055,6057,6059,6061,6063,6065,6067,6069,6071,6073,6075,6077,6079,6081,6083,6085,6087,6089,6091,6093,6095,6097,6099,6101,6103,6105,6107,6109,6111,6113,6115,6117,6119,6121,6123,6125,6127,6129,6131,6133,6135,6137,6139,6141,6143,6145,6147,6149,6151,6153,6155,6157,6159,6161,6163,6165,6167,6169,6171,6173,6175,6177,6179,6181,6183,6185,6187,6189,6191,6193,6195,6197,6199,6201,6203,6205,6207,6209,6211,6213,6215,6217,6219,6221,6223,6225,6227,6229,6231,6233,6235,6237,6239,6241,6243,6245,6247,6249,6251,6253,6255,6257,6259,6261,6263,6265,6267,6269,6271,6273,6275,6277,6279,6281,6283,6285,6287,6289,6291,6293,6295,6297,6299,6301,6303,6305,6307,6309,6311,6313,6315,6317,6319,6321,6323,6325,6327,6329,6331,6333,6335,6337,6339,6341,6343,6345,6347,6349,6351,6353,6355,6357,6359,6361,6363,6365,6367,6369,6371,6373,6375,6377,6379,6381,6383,6385,6387,6389,6391,6393,6395,6397,6399,6401,6403,6405,6407,6409,6411,6413,6415,6417,6419,6421,6423,6425,6427,6429,6431,6433,6435,6437,6439,6441,6443,6445,6447,6449,6451,6453,6455,6457,6459,6461,6463,6465,6467,6469,6471,6473,6475,6477,6479,6481,6483,6485,6487,6489,6491,6493,6495,6497,6499,6501,6503,6505,6507,6509,6511,6513,6515,6517,6519,6521,6523,6525,6527,6529,6531,6533,6535,6537,6539,6541,6543,6545,6547,6549,6551,6553,6555,6557,6559,6561,6563,6565,6567,6569,6571,6573,6575,6577,6579,6581,6583,6585,6587,6589,6591,6593,6595,6597,6599,6601,6603,6605,6607,6609,6611,6613,6615,6617,6619,6621,6623,6625,6627,6629,6631,6633,6635,6637,6639,6641,6643,6645,6647,6649,6651,6653,6655,6657,6659,6661,6663,6665,6667,6669,6671,6673,6675,6677,6679,6681,6683,6685,6687,6689,6691,6693,6695,6697,6699,6701,6703,6705,6707,6709,6711,6713,6715,6717,6719,6721,6723,6725,6727,6729,6731,6733,6735,6737,6739,6741,6743,6745,6747,6749,6751,6753,6755,6757,6759,6761,6763,6765,6767,6769,6771,6773,6775,6777,6779,6781,6783,6785,6787,6789,6791,6793,6795,6797,6799,6801,6803,6805,6807,6809,6811,6813,6815,6817,6819,6821,6823,6825,6827,6829,6831,6833,6835,6837,6839,6841,6843,6845,6847,6849,6851,6853,6855,6857,6859,6861,6863,6865,6867,6869,6871,6873,6875,6877,6879,6881,6883,6885,6887,6889,6891,6893,6895,6897,6899,6901,6903,6905,6907,6909,6911,6913,6915,6917,6919,6921,6923,6925,6927,6929,6931,6933,6935,6937,6939,6941,6943,6945,6947,6949,6951,6953,6955,6957,6959,6961,6963,6965,6967,6969,6971,6973,6975,6977,6979,6981,6983,6985,6987,6989,6991,6993,6995,6997,6999,7001,7003,7005,7007,7009,7011,7013,7015,7017,7019,7021,7023,7025,7027,7029,7031,7033,7035,7037,7039,7041,7043,7045,7047,7049,7051,7053,7055,7057,7059,7061,7063,7065,7067,7069,7071,7073,7075],{"categories":175},[117],{"categories":177},[178],"Developer Productivity",{"categories":180},[117],{"categories":182},[183],"Business & SaaS",{"categories":185},[117],{"categories":187},[188],"AI Automation",{"categories":190},[191],"Product Strategy",{"categories":193},[188],{"categories":195},[117],{"categories":197},[178],{"categories":199},[188],{"categories":201},[202],"Software Engineering",{"categories":204},[117],{"categories":206},[183],{"categories":208},[],{"categories":210},[117],{"categories":212},[213],"Inference & Serving",{"categories":215},[117],{"categories":217},[117],{"categories":219},[188],{"categories":221},[],{"categories":223},[224],"AI News & Trends",{"categories":226},[227],"Data Science & Visualization",{"categories":229},[188],{"categories":231},[117],{"categories":233},[117],{"categories":235},[183],{"categories":237},[178],{"categories":239},[117],{"categories":241},[188],{"categories":243},[224],{"categories":245},[117],{"categories":247},[188],{"categories":249},[188],{"categories":251},[117],{"categories":253},[117],{"categories":255},[188],{"categories":257},[117],{"categories":259},[117],{"categories":261},[117],{"categories":263},[188],{"categories":265},[224],{"categories":267},[117],{"categories":269},[117],{"categories":271},[117],{"categories":273},[],{"categories":275},[276],"Design & Frontend",{"categories":278},[227],{"categories":280},[224],{"categories":282},[117],{"categories":284},[117],{"categories":286},[117],{"categories":288},[],{"categories":290},[117],{"categories":292},[117],{"categories":294},[188],{"categories":296},[202],{"categories":298},[117],{"categories":300},[188],{"categories":302},[117],{"categories":304},[305],"Marketing & Growth",{"categories":307},[276],{"categories":309},[117],{"categories":311},[188],{"categories":313},[117],{"categories":315},[117],{"categories":317},[202],{"categories":319},[117],{"categories":321},[],{"categories":323},[],{"categories":325},[276],{"categories":327},[117],{"categories":329},[188],{"categories":331},[178],{"categories":333},[202],{"categories":335},[188],{"categories":337},[276],{"categories":339},[191],{"categories":341},[117],{"categories":343},[202],{"categories":345},[346],"DevOps & Cloud",{"categories":348},[188],{"categories":350},[191],{"categories":352},[224],{"categories":354},[117],{"categories":356},[],{"categories":358},[117],{"categories":360},[117],{"categories":362},[],{"categories":364},[188],{"categories":366},[202],{"categories":368},[],{"categories":370},[202],{"categories":372},[117],{"categories":374},[375],"Governance & Standards",{"categories":377},[183],{"categories":379},[],{"categories":381},[],{"categories":383},[117],{"categories":385},[117],{"categories":387},[188],{"categories":389},[117],{"categories":391},[117],{"categories":393},[188],{"categories":395},[117],{"categories":397},[117],{"categories":399},[117],{"categories":401},[],{"categories":403},[202],{"categories":405},[],{"categories":407},[],{"categories":409},[117],{"categories":411},[202],{"categories":413},[],{"categories":415},[202],{"categories":417},[117],{"categories":419},[117],{"categories":421},[305],{"categories":423},[117],{"categories":425},[117],{"categories":427},[117],{"categories":429},[276],{"categories":431},[276],{"categories":433},[117],{"categories":435},[202],{"categories":437},[188],{"categories":439},[440],"GovTech & Public-Sector Adoption",{"categories":442},[202],{"categories":444},[117],{"categories":446},[117],{"categories":448},[117],{"categories":450},[188],{"categories":452},[188],{"categories":454},[227],{"categories":456},[117],{"categories":458},[224],{"categories":460},[188],{"categories":462},[463],"Legal AI Tools",{"categories":465},[117],{"categories":467},[188],{"categories":469},[117],{"categories":471},[305],{"categories":473},[188],{"categories":475},[191],{"categories":477},[117],{"categories":479},[202],{"categories":481},[440],{"categories":483},[],{"categories":485},[188],{"categories":487},[],{"categories":489},[183],{"categories":491},[188],{"categories":493},[188],{"categories":495},[496],"RAG & Retrieval",{"categories":498},[183],{"categories":500},[117],{"categories":502},[202],{"categories":504},[202],{"categories":506},[346],{"categories":508},[276],{"categories":510},[188],{"categories":512},[117],{"categories":514},[117],{"categories":516},[],{"categories":518},[519],"Agents & Orchestration",{"categories":521},[202],{"categories":523},[117],{"categories":525},[],{"categories":527},[188],{"categories":529},[183],{"categories":531},[],{"categories":533},[117],{"categories":535},[],{"categories":537},[117],{"categories":539},[178],{"categories":541},[202],{"categories":543},[183],{"categories":545},[117],{"categories":547},[188],{"categories":549},[117],{"categories":551},[224],{"categories":553},[117],{"categories":555},[],{"categories":557},[117],{"categories":559},[],{"categories":561},[117],{"categories":563},[202],{"categories":565},[117],{"categories":567},[188],{"categories":569},[227],{"categories":571},[],{"categories":573},[117],{"categories":575},[276],{"categories":577},[578],"Models & Frontier Labs",{"categories":580},[],{"categories":582},[276],{"categories":584},[585],"Regulation & Governance of AI",{"categories":587},[191],{"categories":589},[188],{"categories":591},[],{"categories":593},[117],{"categories":595},[117],{"categories":597},[188],{"categories":599},[188],{"categories":601},[224],{"categories":603},[117],{"categories":605},[183],{"categories":607},[117],{"categories":609},[188],{"categories":611},[],{"categories":613},[202],{"categories":615},[188],{"categories":617},[117],{"categories":619},[191],{"categories":621},[117],{"categories":623},[624],"AI Policy & Regulation",{"categories":626},[],{"categories":628},[117],{"categories":630},[188],{"categories":632},[191],{"categories":634},[188],{"categories":636},[117],{"categories":638},[117],{"categories":640},[117],{"categories":642},[188],{"categories":644},[],{"categories":646},[227],{"categories":648},[649],"Evals & Reliability",{"categories":651},[117],{"categories":653},[117],{"categories":655},[],{"categories":657},[178],{"categories":659},[440],{"categories":661},[624],{"categories":663},[117],{"categories":665},[183],{"categories":667},[117],{"categories":669},[188],{"categories":671},[117],{"categories":673},[188],{"categories":675},[519],{"categories":677},[117],{"categories":679},[202],{"categories":681},[117],{"categories":683},[],{"categories":685},[276],{"categories":687},[],{"categories":689},[117],{"categories":691},[440],{"categories":693},[117],{"categories":695},[117],{"categories":697},[117],{"categories":699},[],{"categories":701},[117],{"categories":703},[276],{"categories":705},[202],{"categories":707},[],{"categories":709},[117],{"categories":711},[],{"categories":713},[188],{"categories":715},[117],{"categories":717},[276],{"categories":719},[],{"categories":721},[117],{"categories":723},[117],{"categories":725},[227],{"categories":727},[188],{"categories":729},[117],{"categories":731},[183],{"categories":733},[188],{"categories":735},[117],{"categories":737},[117],{"categories":739},[202],{"categories":741},[276],{"categories":743},[117],{"categories":745},[188],{"categories":747},[],{"categories":749},[202],{"categories":751},[188],{"categories":753},[227],{"categories":755},[],{"categories":757},[117],{"categories":759},[224],{"categories":761},[117],{"categories":763},[],{"categories":765},[117],{"categories":767},[117],{"categories":769},[117],{"categories":771},[183,305],{"categories":773},[],{"categories":775},[202],{"categories":777},[117],{"categories":779},[117],{"categories":781},[188],{"categories":783},[117],{"categories":785},[],{"categories":787},[],{"categories":789},[117],{"categories":791},[276],{"categories":793},[117],{"categories":795},[],{"categories":797},[117],{"categories":799},[346],{"categories":801},[],{"categories":803},[188],{"categories":805},[224],{"categories":807},[117],{"categories":809},[117],{"categories":811},[276],{"categories":813},[],{"categories":815},[224],{"categories":817},[117],{"categories":819},[213],{"categories":821},[117],{"categories":823},[117],{"categories":825},[188],{"categories":827},[224],{"categories":829},[578],{"categories":831},[117],{"categories":833},[305],{"categories":835},[],{"categories":837},[188],{"categories":839},[183],{"categories":841},[202],{"categories":843},[117],{"categories":845},[188],{"categories":847},[],{"categories":849},[117,346],{"categories":851},[117],{"categories":853},[117],{"categories":855},[117],{"categories":857},[188],{"categories":859},[117,202],{"categories":861},[227],{"categories":863},[117],{"categories":865},[117],{"categories":867},[117],{"categories":869},[202],{"categories":871},[117],{"categories":873},[188],{"categories":875},[188],{"categories":877},[624],{"categories":879},[305],{"categories":881},[117],{"categories":883},[188],{"categories":885},[117],{"categories":887},[117],{"categories":889},[188],{"categories":891},[],{"categories":893},[188],{"categories":895},[117],{"categories":897},[117],{"categories":899},[188],{"categories":901},[117],{"categories":903},[117,183],{"categories":905},[117],{"categories":907},[183],{"categories":909},[],{"categories":911},[276],{"categories":913},[276],{"categories":915},[117],{"categories":917},[],{"categories":919},[],{"categories":921},[117],{"categories":923},[224],{"categories":925},[],{"categories":927},[178],{"categories":929},[117],{"categories":931},[202],{"categories":933},[117],{"categories":935},[936],"Generative UI & Design-to-Code",{"categories":938},[117],{"categories":940},[117],{"categories":942},[276],{"categories":944},[117],{"categories":946},[947],"Algorithmic Accountability",{"categories":949},[188],{"categories":951},[202],{"categories":953},[224],{"categories":955},[276],{"categories":957},[117],{"categories":959},[],{"categories":961},[191],{"categories":963},[117],{"categories":965},[117],{"categories":967},[117],{"categories":969},[117],{"categories":971},[188],{"categories":973},[974],"MLOps & Infrastructure",{"categories":976},[117],{"categories":978},[117],{"categories":980},[117],{"categories":982},[117],{"categories":984},[117],{"categories":986},[202],{"categories":988},[224],{"categories":990},[117],{"categories":992},[191],{"categories":994},[178],{"categories":996},[117],{"categories":998},[188],{"categories":1000},[346],{"categories":1002},[117],{"categories":1004},[183],{"categories":1006},[117],{"categories":1008},[276],{"categories":1010},[117],{"categories":1012},[117],{"categories":1014},[188],{"categories":1016},[],{"categories":1018},[],{"categories":1020},[117],{"categories":1022},[213],{"categories":1024},[276],{"categories":1026},[224],{"categories":1028},[227],{"categories":1030},[],{"categories":1032},[117],{"categories":1034},[117],{"categories":1036},[183],{"categories":1038},[188],{"categories":1040},[117],{"categories":1042},[117],{"categories":1044},[117],{"categories":1046},[117],{"categories":1048},[224],{"categories":1050},[213],{"categories":1052},[117],{"categories":1054},[276],{"categories":1056},[117],{"categories":1058},[],{"categories":1060},[188],{"categories":1062},[202],{"categories":1064},[],{"categories":1066},[117],{"categories":1068},[117],{"categories":1070},[188],{"categories":1072},[202],{"categories":1074},[117],{"categories":1076},[227],{"categories":1078},[276],{"categories":1080},[],{"categories":1082},[117],{"categories":1084},[],{"categories":1086},[117],{"categories":1088},[],{"categories":1090},[117],{"categories":1092},[117],{"categories":1094},[191],{"categories":1096},[183],{"categories":1098},[188],{"categories":1100},[188],{"categories":1102},[],{"categories":1104},[117],{"categories":1106},[178],{"categories":1108},[117],{"categories":1110},[117],{"categories":1112},[183],{"categories":1114},[224],{"categories":1116},[178],{"categories":1118},[],{"categories":1120},[117],{"categories":1122},[],{"categories":1124},[117],{"categories":1126},[],{"categories":1128},[224],{"categories":1130},[224],{"categories":1132},[],{"categories":1134},[519],{"categories":1136},[117],{"categories":1138},[276],{"categories":1140},[202],{"categories":1142},[],{"categories":1144},[463],{"categories":1146},[188],{"categories":1148},[183],{"categories":1150},[],{"categories":1152},[],{"categories":1154},[178],{"categories":1156},[227],{"categories":1158},[],{"categories":1160},[305],{"categories":1162},[188],{"categories":1164},[183],{"categories":1166},[188],{"categories":1168},[117],{"categories":1170},[183],{"categories":1172},[117],{"categories":1174},[202],{"categories":1176},[],{"categories":1178},[213],{"categories":1180},[191],{"categories":1182},[117],{"categories":1184},[276],{"categories":1186},[202],{"categories":1188},[183],{"categories":1190},[117],{"categories":1192},[202],{"categories":1194},[117],{"categories":1196},[188],{"categories":1198},[183],{"categories":1200},[117],{"categories":1202},[117],{"categories":1204},[117],{"categories":1206},[117],{"categories":1208},[117],{"categories":1210},[],{"categories":1212},[],{"categories":1214},[202],{"categories":1216},[227],{"categories":1218},[191],{"categories":1220},[117],{"categories":1222},[188],{"categories":1224},[202],{"categories":1226},[202],{"categories":1228},[117],{"categories":1230},[],{"categories":1232},[224],{"categories":1234},[191],{"categories":1236},[191],{"categories":1238},[202],{"categories":1240},[117],{"categories":1242},[649],{"categories":1244},[346],{"categories":1246},[],{"categories":1248},[188],{"categories":1250},[117],{"categories":1252},[],{"categories":1254},[178],{"categories":1256},[],{"categories":1258},[117],{"categories":1260},[117],{"categories":1262},[117],{"categories":1264},[276],{"categories":1266},[305],{"categories":1268},[117],{"categories":1270},[202],{"categories":1272},[117],{"categories":1274},[188],{"categories":1276},[],{"categories":1278},[202],{"categories":1280},[117],{"categories":1282},[178],{"categories":1284},[],{"categories":1286},[183],{"categories":1288},[117],{"categories":1290},[117],{"categories":1292},[224],{"categories":1294},[117,346],{"categories":1296},[117],{"categories":1298},[1299],"Design Systems for AI",{"categories":1301},[117],{"categories":1303},[117],{"categories":1305},[224],{"categories":1307},[117],{"categories":1309},[117],{"categories":1311},[117],{"categories":1313},[183],{"categories":1315},[117],{"categories":1317},[117],{"categories":1319},[117],{"categories":1321},[],{"categories":1323},[117],{"categories":1325},[117],{"categories":1327},[183],{"categories":1329},[117],{"categories":1331},[],{"categories":1333},[188],{"categories":1335},[188],{"categories":1337},[202],{"categories":1339},[224],{"categories":1341},[202],{"categories":1343},[117],{"categories":1345},[276],{"categories":1347},[224],{"categories":1349},[227],{"categories":1351},[117],{"categories":1353},[117],{"categories":1355},[188],{"categories":1357},[178],{"categories":1359},[624],{"categories":1361},[117],{"categories":1363},[188],{"categories":1365},[117],{"categories":1367},[202],{"categories":1369},[202],{"categories":1371},[],{"categories":1373},[],{"categories":1375},[117],{"categories":1377},[188],{"categories":1379},[191],{"categories":1381},[],{"categories":1383},[183],{"categories":1385},[117],{"categories":1387},[],{"categories":1389},[276],{"categories":1391},[202],{"categories":1393},[188],{"categories":1395},[202],{"categories":1397},[276],{"categories":1399},[117],{"categories":1401},[117],{"categories":1403},[276],{"categories":1405},[],{"categories":1407},[],{"categories":1409},[224],{"categories":1411},[188],{"categories":1413},[188],{"categories":1415},[117],{"categories":1417},[117],{"categories":1419},[117],{"categories":1421},[117],{"categories":1423},[183],{"categories":1425},[117],{"categories":1427},[117],{"categories":1429},[],{"categories":1431},[202],{"categories":1433},[202],{"categories":1435},[117],{"categories":1437},[202],{"categories":1439},[183],{"categories":1441},[],{"categories":1443},[117],{"categories":1445},[117],{"categories":1447},[117],{"categories":1449},[117],{"categories":1451},[117],{"categories":1453},[188],{"categories":1455},[178],{"categories":1457},[183],{"categories":1459},[117],{"categories":1461},[188],{"categories":1463},[224],{"categories":1465},[188],{"categories":1467},[213],{"categories":1469},[305],{"categories":1471},[117],{"categories":1473},[188],{"categories":1475},[117],{"categories":1477},[117],{"categories":1479},[117],{"categories":1481},[],{"categories":1483},[276],{"categories":1485},[],{"categories":1487},[117],{"categories":1489},[117],{"categories":1491},[],{"categories":1493},[117],{"categories":1495},[202],{"categories":1497},[183],{"categories":1499},[1500],"Visual & Generative Media",{"categories":1502},[188],{"categories":1504},[],{"categories":1506},[117],{"categories":1508},[117],{"categories":1510},[202],{"categories":1512},[346],{"categories":1514},[117],{"categories":1516},[227],{"categories":1518},[624],{"categories":1520},[202],{"categories":1522},[305],{"categories":1524},[117],{"categories":1526},[276],{"categories":1528},[117],{"categories":1530},[117],{"categories":1532},[202],{"categories":1534},[188],{"categories":1536},[117],{"categories":1538},[],{"categories":1540},[],{"categories":1542},[188],{"categories":1544},[202],{"categories":1546},[178],{"categories":1548},[188],{"categories":1550},[578],{"categories":1552},[117],{"categories":1554},[191],{"categories":1556},[117],{"categories":1558},[183],{"categories":1560},[],{"categories":1562},[117],{"categories":1564},[191],{"categories":1566},[117],{"categories":1568},[117],{"categories":1570},[117],{"categories":1572},[191],{"categories":1574},[117],{"categories":1576},[117],{"categories":1578},[305],{"categories":1580},[117],{"categories":1582},[519],{"categories":1584},[117],{"categories":1586},[188],{"categories":1588},[117],{"categories":1590},[117],{"categories":1592},[117],{"categories":1594},[117],{"categories":1596},[276],{"categories":1598},[188],{"categories":1600},[],{"categories":1602},[188],{"categories":1604},[],{"categories":1606},[346],{"categories":1608},[202],{"categories":1610},[],{"categories":1612},[578],{"categories":1614},[117],{"categories":1616},[188],{"categories":1618},[188],{"categories":1620},[117],{"categories":1622},[276,117],{"categories":1624},[178],{"categories":1626},[117],{"categories":1628},[276],{"categories":1630},[],{"categories":1632},[117],{"categories":1634},[178],{"categories":1636},[117],{"categories":1638},[1639],"Medical Imaging & Radiology",{"categories":1641},[117],{"categories":1643},[117],{"categories":1645},[117],{"categories":1647},[276],{"categories":1649},[188],{"categories":1651},[202],{"categories":1653},[],{"categories":1655},[117],{"categories":1657},[117],{"categories":1659},[117],{"categories":1661},[],{"categories":1663},[],{"categories":1665},[117],{"categories":1667},[117],{"categories":1669},[519],{"categories":1671},[117],{"categories":1673},[178],{"categories":1675},[117],{"categories":1677},[117],{"categories":1679},[],{"categories":1681},[188],{"categories":1683},[117],{"categories":1685},[191],{"categories":1687},[202],{"categories":1689},[117],{"categories":1691},[188],{"categories":1693},[519],{"categories":1695},[117],{"categories":1697},[188],{"categories":1699},[117],{"categories":1701},[117],{"categories":1703},[117],{"categories":1705},[276],{"categories":1707},[188],{"categories":1709},[346],{"categories":1711},[276],{"categories":1713},[183],{"categories":1715},[188],{"categories":1717},[224],{"categories":1719},[117],{"categories":1721},[117],{"categories":1723},[191],{"categories":1725},[117],{"categories":1727},[117],{"categories":1729},[117],{"categories":1731},[117],{"categories":1733},[188],{"categories":1735},[117],{"categories":1737},[202],{"categories":1739},[202],{"categories":1741},[117],{"categories":1743},[191],{"categories":1745},[],{"categories":1747},[224],{"categories":1749},[],{"categories":1751},[191],{"categories":1753},[188],{"categories":1755},[117],{"categories":1757},[188],{"categories":1759},[1299],{"categories":1761},[1299],{"categories":1763},[276],{"categories":1765},[117],{"categories":1767},[117],{"categories":1769},[117],{"categories":1771},[188],{"categories":1773},[202],{"categories":1775},[276],{"categories":1777},[188],{"categories":1779},[224],{"categories":1781},[],{"categories":1783},[117],{"categories":1785},[],{"categories":1787},[117],{"categories":1789},[117],{"categories":1791},[117],{"categories":1793},[117],{"categories":1795},[188],{"categories":1797},[1798],"Contract Review & E-Discovery",{"categories":1800},[117],{"categories":1802},[276],{"categories":1804},[117],{"categories":1806},[178],{"categories":1808},[117],{"categories":1810},[224],{"categories":1812},[117],{"categories":1814},[117],{"categories":1816},[305],{"categories":1818},[202],{"categories":1820},[117],{"categories":1822},[117],{"categories":1824},[188],{"categories":1826},[188],{"categories":1828},[947],{"categories":1830},[117],{"categories":1832},[117],{"categories":1834},[188],{"categories":1836},[188],{"categories":1838},[117],{"categories":1840},[117],{"categories":1842},[117],{"categories":1844},[188],{"categories":1846},[117],{"categories":1848},[117],{"categories":1850},[519],{"categories":1852},[496],{"categories":1854},[117],{"categories":1856},[188],{"categories":1858},[117],{"categories":1860},[1861],"Law-Firm Practice & Adoption",{"categories":1863},[117],{"categories":1865},[188],{"categories":1867},[276],{"categories":1869},[117],{"categories":1871},[117],{"categories":1873},[117],{"categories":1875},[],{"categories":1877},[],{"categories":1879},[202],{"categories":1881},[117],{"categories":1883},[],{"categories":1885},[188],{"categories":1887},[178],{"categories":1889},[346],{"categories":1891},[117],{"categories":1893},[],{"categories":1895},[178],{"categories":1897},[183],{"categories":1899},[117],{"categories":1901},[305],{"categories":1903},[],{"categories":1905},[183],{"categories":1907},[188],{"categories":1909},[183],{"categories":1911},[],{"categories":1913},[117],{"categories":1915},[191],{"categories":1917},[117],{"categories":1919},[202],{"categories":1921},[],{"categories":1923},[],{"categories":1925},[],{"categories":1927},[],{"categories":1929},[117],{"categories":1931},[191],{"categories":1933},[188],{"categories":1935},[346],{"categories":1937},[117],{"categories":1939},[178],{"categories":1941},[202],{"categories":1943},[117],{"categories":1945},[117],{"categories":1947},[202],{"categories":1949},[191],{"categories":1951},[117],{"categories":1953},[117],{"categories":1955},[117],{"categories":1957},[974],{"categories":1959},[117],{"categories":1961},[202],{"categories":1963},[117],{"categories":1965},[305],{"categories":1967},[202],{"categories":1969},[183],{"categories":1971},[117],{"categories":1973},[117],{"categories":1975},[117],{"categories":1977},[276],{"categories":1979},[117],{"categories":1981},[117],{"categories":1983},[117],{"categories":1985},[117],{"categories":1987},[183],{"categories":1989},[188],{"categories":1991},[117,178],{"categories":1993},[519],{"categories":1995},[117],{"categories":1997},[117],{"categories":1999},[202],{"categories":2001},[202],{"categories":2003},[276],{"categories":2005},[188],{"categories":2007},[188],{"categories":2009},[202],{"categories":2011},[117],{"categories":2013},[117],{"categories":2015},[117],{"categories":2017},[],{"categories":2019},[],{"categories":2021},[117],{"categories":2023},[227],{"categories":2025},[117],{"categories":2027},[188],{"categories":2029},[],{"categories":2031},[117],{"categories":2033},[117],{"categories":2035},[202],{"categories":2037},[227],{"categories":2039},[224],{"categories":2041},[276],{"categories":2043},[117],{"categories":2045},[188],{"categories":2047},[117],{"categories":2049},[202],{"categories":2051},[],{"categories":2053},[188],{"categories":2055},[117],{"categories":2057},[117],{"categories":2059},[117],{"categories":2061},[117],{"categories":2063},[],{"categories":2065},[188],{"categories":2067},[117],{"categories":2069},[117],{"categories":2071},[117],{"categories":2073},[],{"categories":2075},[188],{"categories":2077},[117],{"categories":2079},[117],{"categories":2081},[183],{"categories":2083},[117],{"categories":2085},[117],{"categories":2087},[],{"categories":2089},[178],{"categories":2091},[117],{"categories":2093},[117],{"categories":2095},[117],{"categories":2097},[276],{"categories":2099},[117],{"categories":2101},[202],{"categories":2103},[117],{"categories":2105},[178],{"categories":2107},[117],{"categories":2109},[202],{"categories":2111},[305],{"categories":2113},[188],{"categories":2115},[188],{"categories":2117},[117],{"categories":2119},[117],{"categories":2121},[117,276],{"categories":2123},[117],{"categories":2125},[188],{"categories":2127},[224],{"categories":2129},[117],{"categories":2131},[224],{"categories":2133},[188],{"categories":2135},[276],{"categories":2137},[117],{"categories":2139},[],{"categories":2141},[202],{"categories":2143},[346],{"categories":2145},[276],{"categories":2147},[202],{"categories":2149},[117],{"categories":2151},[191],{"categories":2153},[117],{"categories":2155},[117],{"categories":2157},[188],{"categories":2159},[],{"categories":2161},[],{"categories":2163},[117],{"categories":2165},[],{"categories":2167},[],{"categories":2169},[191],{"categories":2171},[202],{"categories":2173},[117],{"categories":2175},[188],{"categories":2177},[188],{"categories":2179},[183],{"categories":2181},[188],{"categories":2183},[346],{"categories":2185},[117],{"categories":2187},[117],{"categories":2189},[117],{"categories":2191},[213],{"categories":2193},[117],{"categories":2195},[117],{"categories":2197},[117],{"categories":2199},[202],{"categories":2201},[188],{"categories":2203},[117],{"categories":2205},[117],{"categories":2207},[202],{"categories":2209},[463],{"categories":2211},[188],{"categories":2213},[947],{"categories":2215},[],{"categories":2217},[276],{"categories":2219},[1861],{"categories":2221},[202],{"categories":2223},[],{"categories":2225},[],{"categories":2227},[117],{"categories":2229},[188],{"categories":2231},[],{"categories":2233},[],{"categories":2235},[117],{"categories":2237},[305],{"categories":2239},[117],{"categories":2241},[305],{"categories":2243},[188],{"categories":2245},[117],{"categories":2247},[117],{"categories":2249},[202],{"categories":2251},[191],{"categories":2253},[],{"categories":2255},[117],{"categories":2257},[117],{"categories":2259},[202],{"categories":2261},[1798],{"categories":2263},[276],{"categories":2265},[276],{"categories":2267},[117],{"categories":2269},[188],{"categories":2271},[178],{"categories":2273},[117],{"categories":2275},[117],{"categories":2277},[117],{"categories":2279},[117],{"categories":2281},[276],{"categories":2283},[276],{"categories":2285},[188],{"categories":2287},[188],{"categories":2289},[188],{"categories":2291},[117],{"categories":2293},[117],{"categories":2295},[],{"categories":2297},[117],{"categories":2299},[],{"categories":2301},[2302],"Interaction & Product Design",{"categories":2304},[117],{"categories":2306},[188],{"categories":2308},[202],{"categories":2310},[375],{"categories":2312},[224],{"categories":2314},[202],{"categories":2316},[117],{"categories":2318},[117],{"categories":2320},[117],{"categories":2322},[202],{"categories":2324},[117],{"categories":2326},[178],{"categories":2328},[188],{"categories":2330},[117],{"categories":2332},[],{"categories":2334},[188],{"categories":2336},[188],{"categories":2338},[188],{"categories":2340},[],{"categories":2342},[202],{"categories":2344},[117],{"categories":2346},[188],{"categories":2348},[178],{"categories":2350},[2302],{"categories":2352},[117],{"categories":2354},[178],{"categories":2356},[178],{"categories":2358},[],{"categories":2360},[188],{"categories":2362},[202],{"categories":2364},[],{"categories":2366},[188],{"categories":2368},[224],{"categories":2370},[117],{"categories":2372},[188],{"categories":2374},[117],{"categories":2376},[188],{"categories":2378},[188],{"categories":2380},[117],{"categories":2382},[117],{"categories":2384},[224],{"categories":2386},[227],{"categories":2388},[117],{"categories":2390},[191],{"categories":2392},[202],{"categories":2394},[2395],"Coding Agents & Dev Productivity",{"categories":2397},[224],{"categories":2399},[276],{"categories":2401},[117],{"categories":2403},[117],{"categories":2405},[],{"categories":2407},[117],{"categories":2409},[947],{"categories":2411},[],{"categories":2413},[117],{"categories":2415},[117],{"categories":2417},[346],{"categories":2419},[117],{"categories":2421},[224],{"categories":2423},[],{"categories":2425},[],{"categories":2427},[117],{"categories":2429},[],{"categories":2431},[188],{"categories":2433},[117],{"categories":2435},[],{"categories":2437},[202],{"categories":2439},[202],{"categories":2441},[117],{"categories":2443},[227],{"categories":2445},[],{"categories":2447},[117],{"categories":2449},[117],{"categories":2451},[117],{"categories":2453},[227],{"categories":2455},[202],{"categories":2457},[188],{"categories":2459},[],{"categories":2461},[],{"categories":2463},[117],{"categories":2465},[117],{"categories":2467},[188],{"categories":2469},[188],{"categories":2471},[440],{"categories":2473},[202],{"categories":2475},[202],{"categories":2477},[188],{"categories":2479},[224],{"categories":2481},[224],{"categories":2483},[188],{"categories":2485},[188],{"categories":2487},[117],{"categories":2489},[178],{"categories":2491},[2302],{"categories":2493},[191],{"categories":2495},[117,346],{"categories":2497},[227],{"categories":2499},[],{"categories":2501},[276],{"categories":2503},[188],{"categories":2505},[202],{"categories":2507},[178],{"categories":2509},[117],{"categories":2511},[188],{"categories":2513},[2514],"The Designer's Role & Craft",{"categories":2516},[276],{"categories":2518},[],{"categories":2520},[188],{"categories":2522},[117],{"categories":2524},[188],{"categories":2526},[188],{"categories":2528},[117],{"categories":2530},[305],{"categories":2532},[117],{"categories":2534},[202],{"categories":2536},[117],{"categories":2538},[276],{"categories":2540},[117],{"categories":2542},[],{"categories":2544},[188],{"categories":2546},[276],{"categories":2548},[191],{"categories":2550},[117],{"categories":2552},[117],{"categories":2554},[117],{"categories":2556},[2557],"AI UX Patterns",{"categories":2559},[188],{"categories":2561},[188],{"categories":2563},[188],{"categories":2565},[188],{"categories":2567},[305],{"categories":2569},[227],{"categories":2571},[117],{"categories":2573},[188],{"categories":2575},[117],{"categories":2577},[1299],{"categories":2579},[],{"categories":2581},[305],{"categories":2583},[188],{"categories":2585},[224],{"categories":2587},[202],{"categories":2589},[117],{"categories":2591},[188],{"categories":2593},[],{"categories":2595},[],{"categories":2597},[117],{"categories":2599},[117],{"categories":2601},[188],{"categories":2603},[117],{"categories":2605},[188],{"categories":2607},[440],{"categories":2609},[276],{"categories":2611},[117],{"categories":2613},[224],{"categories":2615},[202],{"categories":2617},[117],{"categories":2619},[188],{"categories":2621},[188],{"categories":2623},[],{"categories":2625},[117],{"categories":2627},[],{"categories":2629},[117],{"categories":2631},[],{"categories":2633},[117],{"categories":2635},[117],{"categories":2637},[117],{"categories":2639},[188],{"categories":2641},[202],{"categories":2643},[],{"categories":2645},[],{"categories":2647},[227],{"categories":2649},[213],{"categories":2651},[117],{"categories":2653},[117],{"categories":2655},[117],{"categories":2657},[227],{"categories":2659},[117],{"categories":2661},[117],{"categories":2663},[224],{"categories":2665},[117],{"categories":2667},[117],{"categories":2669},[117],{"categories":2671},[188],{"categories":2673},[117],{"categories":2675},[188],{"categories":2677},[117],{"categories":2679},[117],{"categories":2681},[117],{"categories":2683},[188],{"categories":2685},[],{"categories":2687},[117],{"categories":2689},[],{"categories":2691},[117],{"categories":2693},[117],{"categories":2695},[346],{"categories":2697},[117],{"categories":2699},[],{"categories":2701},[],{"categories":2703},[276],{"categories":2705},[974],{"categories":2707},[188],{"categories":2709},[178],{"categories":2711},[2514],{"categories":2713},[],{"categories":2715},[],{"categories":2717},[117],{"categories":2719},[],{"categories":2721},[],{"categories":2723},[202],{"categories":2725},[224],{"categories":2727},[305],{"categories":2729},[188],{"categories":2731},[183],{"categories":2733},[117],{"categories":2735},[117],{"categories":2737},[183],{"categories":2739},[],{"categories":2741},[276],{"categories":2743},[191],{"categories":2745},[117],{"categories":2747},[117],{"categories":2749},[188],{"categories":2751},[183],{"categories":2753},[117],{"categories":2755},[117],{"categories":2757},[178],{"categories":2759},[117],{"categories":2761},[117],{"categories":2763},[],{"categories":2765},[178],{"categories":2767},[117],{"categories":2769},[305],{"categories":2771},[188],{"categories":2773},[224],{"categories":2775},[117],{"categories":2777},[202],{"categories":2779},[117],{"categories":2781},[117],{"categories":2783},[183],{"categories":2785},[117],{"categories":2787},[117],{"categories":2789},[117],{"categories":2791},[188],{"categories":2793},[117],{"categories":2795},[],{"categories":2797},[117],{"categories":2799},[202],{"categories":2801},[178],{"categories":2803},[117],{"categories":2805},[117],{"categories":2807},[117],{"categories":2809},[],{"categories":2811},[117],{"categories":2813},[519],{"categories":2815},[188],{"categories":2817},[183],{"categories":2819},[224],{"categories":2821},[117],{"categories":2823},[117],{"categories":2825},[],{"categories":2827},[183],{"categories":2829},[183],{"categories":2831},[117],{"categories":2833},[117],{"categories":2835},[191],{"categories":2837},[117],{"categories":2839},[117],{"categories":2841},[117],{"categories":2843},[117],{"categories":2845},[202],{"categories":2847},[202],{"categories":2849},[117],{"categories":2851},[],{"categories":2853},[202],{"categories":2855},[117],{"categories":2857},[202],{"categories":2859},[188],{"categories":2861},[624],{"categories":2863},[],{"categories":2865},[],{"categories":2867},[117],{"categories":2869},[224],{"categories":2871},[],{"categories":2873},[346],{"categories":2875},[117],{"categories":2877},[117],{"categories":2879},[117],{"categories":2881},[276],{"categories":2883},[936],{"categories":2885},[],{"categories":2887},[117],{"categories":2889},[117],{"categories":2891},[117],{"categories":2893},[202],{"categories":2895},[117],{"categories":2897},[117],{"categories":2899},[117,346],{"categories":2901},[117],{"categories":2903},[117],{"categories":2905},[276],{"categories":2907},[188],{"categories":2909},[],{"categories":2911},[188],{"categories":2913},[188],{"categories":2915},[117],{"categories":2917},[117],{"categories":2919},[117],{"categories":2921},[117],{"categories":2923},[227],{"categories":2925},[117],{"categories":2927},[2557],{"categories":2929},[178],{"categories":2931},[227],{"categories":2933},[178],{"categories":2935},[202],{"categories":2937},[276],{"categories":2939},[188],{"categories":2941},[117],{"categories":2943},[],{"categories":2945},[183],{"categories":2947},[117],{"categories":2949},[117],{"categories":2951},[224],{"categories":2953},[117],{"categories":2955},[117],{"categories":2957},[117],{"categories":2959},[188],{"categories":2961},[117],{"categories":2963},[117],{"categories":2965},[117],{"categories":2967},[183],{"categories":2969},[],{"categories":2971},[346],{"categories":2973},[117],{"categories":2975},[440],{"categories":2977},[276],{"categories":2979},[276],{"categories":2981},[202],{"categories":2983},[188],{"categories":2985},[117],{"categories":2987},[183],{"categories":2989},[224],{"categories":2991},[117],{"categories":2993},[117],{"categories":2995},[117],{"categories":2997},[276],{"categories":2999},[188],{"categories":3001},[188],{"categories":3003},[117],{"categories":3005},[117],{"categories":3007},[578],{"categories":3009},[188],{"categories":3011},[],{"categories":3013},[117],{"categories":3015},[117],{"categories":3017},[117],{"categories":3019},[],{"categories":3021},[],{"categories":3023},[117],{"categories":3025},[117],{"categories":3027},[188],{"categories":3029},[117],{"categories":3031},[117],{"categories":3033},[117],{"categories":3035},[202],{"categories":3037},[117],{"categories":3039},[117],{"categories":3041},[188],{"categories":3043},[117],{"categories":3045},[117],{"categories":3047},[117],{"categories":3049},[117],{"categories":3051},[117],{"categories":3053},[],{"categories":3055},[202],{"categories":3057},[227],{"categories":3059},[117],{"categories":3061},[188],{"categories":3063},[188],{"categories":3065},[117],{"categories":3067},[117],{"categories":3069},[],{"categories":3071},[],{"categories":3073},[117],{"categories":3075},[117],{"categories":3077},[117],{"categories":3079},[224],{"categories":3081},[227],{"categories":3083},[],{"categories":3085},[117],{"categories":3087},[276],{"categories":3089},[117],{"categories":3091},[346],{"categories":3093},[1861],{"categories":3095},[224],{"categories":3097},[202],{"categories":3099},[117],{"categories":3101},[202],{"categories":3103},[202],{"categories":3105},[117],{"categories":3107},[117],{"categories":3109},[202],{"categories":3111},[224],{"categories":3113},[224],{"categories":3115},[346],{"categories":3117},[188],{"categories":3119},[],{"categories":3121},[224],{"categories":3123},[117],{"categories":3125},[188],{"categories":3127},[178],{"categories":3129},[202],{"categories":3131},[117],{"categories":3133},[224],{"categories":3135},[],{"categories":3137},[117],{"categories":3139},[202],{"categories":3141},[202],{"categories":3143},[227],{"categories":3145},[117],{"categories":3147},[224],{"categories":3149},[117],{"categories":3151},[202],{"categories":3153},[188],{"categories":3155},[188],{"categories":3157},[224],{"categories":3159},[188],{"categories":3161},[346],{"categories":3163},[188],{"categories":3165},[117],{"categories":3167},[117],{"categories":3169},[117],{"categories":3171},[117],{"categories":3173},[202],{"categories":3175},[117],{"categories":3177},[],{"categories":3179},[188],{"categories":3181},[183],{"categories":3183},[202],{"categories":3185},[],{"categories":3187},[],{"categories":3189},[117],{"categories":3191},[188],{"categories":3193},[117],{"categories":3195},[117],{"categories":3197},[3198],"Frameworks & Tooling",{"categories":3200},[117],{"categories":3202},[117],{"categories":3204},[202],{"categories":3206},[117],{"categories":3208},[117],{"categories":3210},[],{"categories":3212},[227],{"categories":3214},[227],{"categories":3216},[178],{"categories":3218},[117],{"categories":3220},[188],{"categories":3222},[117],{"categories":3224},[276],{"categories":3226},[],{"categories":3228},[1861],{"categories":3230},[117],{"categories":3232},[202],{"categories":3234},[117],{"categories":3236},[346],{"categories":3238},[346],{"categories":3240},[],{"categories":3242},[188],{"categories":3244},[188],{"categories":3246},[117],{"categories":3248},[117],{"categories":3250},[224],{"categories":3252},[188],{"categories":3254},[224],{"categories":3256},[117],{"categories":3258},[188],{"categories":3260},[],{"categories":3262},[276],{"categories":3264},[117],{"categories":3266},[117],{"categories":3268},[],{"categories":3270},[117],{"categories":3272},[188],{"categories":3274},[117],{"categories":3276},[117],{"categories":3278},[117],{"categories":3280},[],{"categories":3282},[202],{"categories":3284},[117],{"categories":3286},[202],{"categories":3288},[346],{"categories":3290},[117],{"categories":3292},[117],{"categories":3294},[117],{"categories":3296},[202],{"categories":3298},[183],{"categories":3300},[117],{"categories":3302},[1861],{"categories":3304},[],{"categories":3306},[188],{"categories":3308},[178],{"categories":3310},[117],{"categories":3312},[178],{"categories":3314},[117],{"categories":3316},[],{"categories":3318},[188],{"categories":3320},[117],{"categories":3322},[117],{"categories":3324},[3325],"AI Design Tooling",{"categories":3327},[276],{"categories":3329},[117],{"categories":3331},[117],{"categories":3333},[202],{"categories":3335},[276],{"categories":3337},[117],{"categories":3339},[117],{"categories":3341},[202],{"categories":3343},[224],{"categories":3345},[191],{"categories":3347},[202],{"categories":3349},[117],{"categories":3351},[117],{"categories":3353},[117],{"categories":3355},[188],{"categories":3357},[117],{"categories":3359},[],{"categories":3361},[188],{"categories":3363},[117],{"categories":3365},[117],{"categories":3367},[188],{"categories":3369},[117],{"categories":3371},[117],{"categories":3373},[117],{"categories":3375},[188],{"categories":3377},[],{"categories":3379},[188],{"categories":3381},[3198],{"categories":3383},[117],{"categories":3385},[117],{"categories":3387},[188],{"categories":3389},[188],{"categories":3391},[202],{"categories":3393},[202],{"categories":3395},[117],{"categories":3397},[],{"categories":3399},[202],{"categories":3401},[117],{"categories":3403},[117],{"categories":3405},[188],{"categories":3407},[183],{"categories":3409},[117],{"categories":3411},[],{"categories":3413},[117],{"categories":3415},[117],{"categories":3417},[2302],{"categories":3419},[],{"categories":3421},[117],{"categories":3423},[117],{"categories":3425},[117],{"categories":3427},[117],{"categories":3429},[276],{"categories":3431},[117],{"categories":3433},[],{"categories":3435},[117],{"categories":3437},[117],{"categories":3439},[117],{"categories":3441},[117],{"categories":3443},[305],{"categories":3445},[224],{"categories":3447},[117],{"categories":3449},[117],{"categories":3451},[1861],{"categories":3453},[178],{"categories":3455},[117],{"categories":3457},[117],{"categories":3459},[227],{"categories":3461},[117],{"categories":3463},[117],{"categories":3465},[224],{"categories":3467},[188],{"categories":3469},[],{"categories":3471},[117],{"categories":3473},[117],{"categories":3475},[276],{"categories":3477},[117],{"categories":3479},[305],{"categories":3481},[188],{"categories":3483},[117],{"categories":3485},[188],{"categories":3487},[],{"categories":3489},[],{"categories":3491},[],{"categories":3493},[178],{"categories":3495},[224],{"categories":3497},[188],{"categories":3499},[117],{"categories":3501},[117],{"categories":3503},[117],{"categories":3505},[117],{"categories":3507},[463],{"categories":3509},[276],{"categories":3511},[188],{"categories":3513},[117],{"categories":3515},[],{"categories":3517},[188],{"categories":3519},[188],{"categories":3521},[],{"categories":3523},[117],{"categories":3525},[188],{"categories":3527},[117],{"categories":3529},[],{"categories":3531},[117],{"categories":3533},[117],{"categories":3535},[117],{"categories":3537},[224],{"categories":3539},[276],{"categories":3541},[188],{"categories":3543},[276],{"categories":3545},[188],{"categories":3547},[117],{"categories":3549},[183],{"categories":3551},[],{"categories":3553},[],{"categories":3555},[117],{"categories":3557},[117],{"categories":3559},[117],{"categories":3561},[178],{"categories":3563},[188],{"categories":3565},[224],{"categories":3567},[],{"categories":3569},[276],{"categories":3571},[],{"categories":3573},[202],{"categories":3575},[117],{"categories":3577},[202],{"categories":3579},[276],{"categories":3581},[202],{"categories":3583},[117],{"categories":3585},[],{"categories":3587},[117],{"categories":3589},[117],{"categories":3591},[],{"categories":3593},[117],{"categories":3595},[117],{"categories":3597},[305],{"categories":3599},[117],{"categories":3601},[117],{"categories":3603},[346],{"categories":3605},[202],{"categories":3607},[117],{"categories":3609},[],{"categories":3611},[188],{"categories":3613},[117],{"categories":3615},[178],{"categories":3617},[578],{"categories":3619},[117],{"categories":3621},[117],{"categories":3623},[188],{"categories":3625},[117],{"categories":3627},[188],{"categories":3629},[117],{"categories":3631},[117],{"categories":3633},[117],{"categories":3635},[117],{"categories":3637},[],{"categories":3639},[117],{"categories":3641},[178],{"categories":3643},[117],{"categories":3645},[183],{"categories":3647},[202],{"categories":3649},[276],{"categories":3651},[],{"categories":3653},[117],{"categories":3655},[],{"categories":3657},[188],{"categories":3659},[117],{"categories":3661},[],{"categories":3663},[188],{"categories":3665},[117],{"categories":3667},[202],{"categories":3669},[276],{"categories":3671},[224],{"categories":3673},[117],{"categories":3675},[224],{"categories":3677},[188],{"categories":3679},[276],{"categories":3681},[117],{"categories":3683},[],{"categories":3685},[117],{"categories":3687},[213],{"categories":3689},[188],{"categories":3691},[117],{"categories":3693},[276],{"categories":3695},[224],{"categories":3697},[183],{"categories":3699},[202],{"categories":3701},[117],{"categories":3703},[117],{"categories":3705},[117],{"categories":3707},[117],{"categories":3709},[224],{"categories":3711},[305],{"categories":3713},[],{"categories":3715},[],{"categories":3717},[227],{"categories":3719},[519],{"categories":3721},[117],{"categories":3723},[188],{"categories":3725},[117,202],{"categories":3727},[224],{"categories":3729},[117],{"categories":3731},[117],{"categories":3733},[117],{"categories":3735},[117],{"categories":3737},[117],{"categories":3739},[117],{"categories":3741},[117],{"categories":3743},[188],{"categories":3745},[117],{"categories":3747},[188],{"categories":3749},[117],{"categories":3751},[117],{"categories":3753},[117],{"categories":3755},[],{"categories":3757},[117],{"categories":3759},[1299],{"categories":3761},[202],{"categories":3763},[276],{"categories":3765},[117],{"categories":3767},[117],{"categories":3769},[117],{"categories":3771},[227],{"categories":3773},[188],{"categories":3775},[305],{"categories":3777},[346],{"categories":3779},[],{"categories":3781},[202],{"categories":3783},[117],{"categories":3785},[183],{"categories":3787},[188],{"categories":3789},[178],{"categories":3791},[188],{"categories":3793},[117],{"categories":3795},[188],{"categories":3797},[188],{"categories":3799},[191],{"categories":3801},[202],{"categories":3803},[117],{"categories":3805},[117],{"categories":3807},[],{"categories":3809},[],{"categories":3811},[],{"categories":3813},[346],{"categories":3815},[117],{"categories":3817},[224],{"categories":3819},[117],{"categories":3821},[117],{"categories":3823},[117],{"categories":3825},[117],{"categories":3827},[],{"categories":3829},[117],{"categories":3831},[227],{"categories":3833},[183],{"categories":3835},[188],{"categories":3837},[117],{"categories":3839},[],{"categories":3841},[117],{"categories":3843},[188],{"categories":3845},[117],{"categories":3847},[346],{"categories":3849},[],{"categories":3851},[276],{"categories":3853},[276],{"categories":3855},[117],{"categories":3857},[188],{"categories":3859},[],{"categories":3861},[202],{"categories":3863},[117],{"categories":3865},[276],{"categories":3867},[117],{"categories":3869},[183],{"categories":3871},[188],{"categories":3873},[117],{"categories":3875},[],{"categories":3877},[224],{"categories":3879},[117],{"categories":3881},[117],{"categories":3883},[117],{"categories":3885},[276],{"categories":3887},[188],{"categories":3889},[224],{"categories":3891},[],{"categories":3893},[188],{"categories":3895},[183],{"categories":3897},[188],{"categories":3899},[276],{"categories":3901},[117],{"categories":3903},[117],{"categories":3905},[117],{"categories":3907},[519],{"categories":3909},[117],{"categories":3911},[188],{"categories":3913},[],{"categories":3915},[117],{"categories":3917},[117],{"categories":3919},[346],{"categories":3921},[224],{"categories":3923},[227],{"categories":3925},[624],{"categories":3927},[227],{"categories":3929},[227],{"categories":3931},[117],{"categories":3933},[],{"categories":3935},[],{"categories":3937},[],{"categories":3939},[188],{"categories":3941},[117],{"categories":3943},[188],{"categories":3945},[188],{"categories":3947},[202],{"categories":3949},[117],{"categories":3951},[496],{"categories":3953},[202],{"categories":3955},[188],{"categories":3957},[117],{"categories":3959},[117],{"categories":3961},[117],{"categories":3963},[117],{"categories":3965},[117],{"categories":3967},[188],{"categories":3969},[117],{"categories":3971},[],{"categories":3973},[],{"categories":3975},[117],{"categories":3977},[],{"categories":3979},[117],{"categories":3981},[188],{"categories":3983},[276],{"categories":3985},[117],{"categories":3987},[117],{"categories":3989},[],{"categories":3991},[188],{"categories":3993},[117],{"categories":3995},[117],{"categories":3997},[191],{"categories":3999},[117],{"categories":4001},[276],{"categories":4003},[117],{"categories":4005},[188],{"categories":4007},[183],{"categories":4009},[117],{"categories":4011},[117],{"categories":4013},[305],{"categories":4015},[188],{"categories":4017},[117],{"categories":4019},[117],{"categories":4021},[936],{"categories":4023},[117],{"categories":4025},[188],{"categories":4027},[117],{"categories":4029},[202],{"categories":4031},[117],{"categories":4033},[578],{"categories":4035},[276],{"categories":4037},[],{"categories":4039},[117],{"categories":4041},[117],{"categories":4043},[224],{"categories":4045},[519],{"categories":4047},[188],{"categories":4049},[117],{"categories":4051},[],{"categories":4053},[224],{"categories":4055},[440],{"categories":4057},[188],{"categories":4059},[188],{"categories":4061},[188],{"categories":4063},[117],{"categories":4065},[117],{"categories":4067},[188],{"categories":4069},[],{"categories":4071},[183],{"categories":4073},[117],{"categories":4075},[183],{"categories":4077},[188],{"categories":4079},[],{"categories":4081},[202],{"categories":4083},[117],{"categories":4085},[117],{"categories":4087},[178],{"categories":4089},[117],{"categories":4091},[224],{"categories":4093},[346],{"categories":4095},[213],{"categories":4097},[188],{"categories":4099},[188],{"categories":4101},[117],{"categories":4103},[117],{"categories":4105},[188],{"categories":4107},[117],{"categories":4109},[178],{"categories":4111},[],{"categories":4113},[188],{"categories":4115},[117],{"categories":4117},[117],{"categories":4119},[117],{"categories":4121},[188],{"categories":4123},[117],{"categories":4125},[],{"categories":4127},[117],{"categories":4129},[],{"categories":4131},[276],{"categories":4133},[188],{"categories":4135},[117,183],{"categories":4137},[188],{"categories":4139},[117],{"categories":4141},[],{"categories":4143},[178],{"categories":4145},[227],{"categories":4147},[183],{"categories":4149},[117],{"categories":4151},[202],{"categories":4153},[117],{"categories":4155},[117],{"categories":4157},[188],{"categories":4159},[117],{"categories":4161},[117],{"categories":4163},[117],{"categories":4165},[224],{"categories":4167},[1299],{"categories":4169},[188],{"categories":4171},[117],{"categories":4173},[],{"categories":4175},[],{"categories":4177},[117],{"categories":4179},[188],{"categories":4181},[117],{"categories":4183},[117],{"categories":4185},[346],{"categories":4187},[],{"categories":4189},[117],{"categories":4191},[188],{"categories":4193},[213],{"categories":4195},[188],{"categories":4197},[519],{"categories":4199},[],{"categories":4201},[463],{"categories":4203},[188],{"categories":4205},[117],{"categories":4207},[117],{"categories":4209},[305],{"categories":4211},[188],{"categories":4213},[117],{"categories":4215},[227],{"categories":4217},[191],{"categories":4219},[188],{"categories":4221},[117],{"categories":4223},[519],{"categories":4225},[117],{"categories":4227},[346],{"categories":4229},[183],{"categories":4231},[],{"categories":4233},[117],{"categories":4235},[117],{"categories":4237},[305],{"categories":4239},[276],{"categories":4241},[117],{"categories":4243},[117],{"categories":4245},[117],{"categories":4247},[],{"categories":4249},[305],{"categories":4251},[224],{"categories":4253},[117],{"categories":4255},[117],{"categories":4257},[117],{"categories":4259},[624],{"categories":4261},[178],{"categories":4263},[117],{"categories":4265},[191],{"categories":4267},[117],{"categories":4269},[],{"categories":4271},[],{"categories":4273},[276],{"categories":4275},[117],{"categories":4277},[227],{"categories":4279},[305],{"categories":4281},[188],{"categories":4283},[117],{"categories":4285},[117],{"categories":4287},[305],{"categories":4289},[224],{"categories":4291},[117],{"categories":4293},[],{"categories":4295},[117],{"categories":4297},[117],{"categories":4299},[],{"categories":4301},[117],{"categories":4303},[117],{"categories":4305},[649],{"categories":4307},[117],{"categories":4309},[117],{"categories":4311},[188],{"categories":4313},[202],{"categories":4315},[519],{"categories":4317},[117],{"categories":4319},[117],{"categories":4321},[117],{"categories":4323},[],{"categories":4325},[117,202],{"categories":4327},[224],{"categories":4329},[188],{"categories":4331},[202],{"categories":4333},[188],{"categories":4335},[974],{"categories":4337},[202],{"categories":4339},[202],{"categories":4341},[188],{"categories":4343},[117],{"categories":4345},[178],{"categories":4347},[],{"categories":4349},[],{"categories":4351},[188],{"categories":4353},[117],{"categories":4355},[202],{"categories":4357},[117],{"categories":4359},[178],{"categories":4361},[202],{"categories":4363},[202],{"categories":4365},[117],{"categories":4367},[305],{"categories":4369},[117],{"categories":4371},[202],{"categories":4373},[117],{"categories":4375},[],{"categories":4377},[117],{"categories":4379},[117],{"categories":4381},[276,117],{"categories":4383},[346],{"categories":4385},[178],{"categories":4387},[117],{"categories":4389},[],{"categories":4391},[117],{"categories":4393},[117],{"categories":4395},[183],{"categories":4397},[117],{"categories":4399},[183],{"categories":4401},[117],{"categories":4403},[117],{"categories":4405},[440],{"categories":4407},[117],{"categories":4409},[183],{"categories":4411},[202],{"categories":4413},[227],{"categories":4415},[188],{"categories":4417},[117],{"categories":4419},[202],{"categories":4421},[117],{"categories":4423},[117],{"categories":4425},[224],{"categories":4427},[305],{"categories":4429},[276],{"categories":4431},[117],{"categories":4433},[117],{"categories":4435},[117],{"categories":4437},[117],{"categories":4439},[178],{"categories":4441},[117],{"categories":4443},[188],{"categories":4445},[188],{"categories":4447},[202],{"categories":4449},[224],{"categories":4451},[202],{"categories":4453},[202],{"categories":4455},[117],{"categories":4457},[117],{"categories":4459},[],{"categories":4461},[],{"categories":4463},[227],{"categories":4465},[117],{"categories":4467},[202],{"categories":4469},[117],{"categories":4471},[276],{"categories":4473},[519],{"categories":4475},[463],{"categories":4477},[440],{"categories":4479},[117],{"categories":4481},[117],{"categories":4483},[117],{"categories":4485},[227],{"categories":4487},[117],{"categories":4489},[117],{"categories":4491},[117],{"categories":4493},[117],{"categories":4495},[117],{"categories":4497},[117],{"categories":4499},[117],{"categories":4501},[188],{"categories":4503},[178],{"categories":4505},[188],{"categories":4507},[117,183],{"categories":4509},[],{"categories":4511},[276],{"categories":4513},[],{"categories":4515},[191],{"categories":4517},[117],{"categories":4519},[224],{"categories":4521},[178],{"categories":4523},[117],{"categories":4525},[178],{"categories":4527},[188],{"categories":4529},[227],{"categories":4531},[188],{"categories":4533},[191],{"categories":4535},[188],{"categories":4537},[117],{"categories":4539},[117],{"categories":4541},[183],{"categories":4543},[188],{"categories":4545},[202],{"categories":4547},[305],{"categories":4549},[117],{"categories":4551},[117],{"categories":4553},[],{"categories":4555},[224],{"categories":4557},[117],{"categories":4559},[117],{"categories":4561},[117],{"categories":4563},[117],{"categories":4565},[117],{"categories":4567},[117],{"categories":4569},[202],{"categories":4571},[224],{"categories":4573},[202],{"categories":4575},[202],{"categories":4577},[117],{"categories":4579},[117],{"categories":4581},[117],{"categories":4583},[117],{"categories":4585},[463],{"categories":4587},[117],{"categories":4589},[188],{"categories":4591},[224],{"categories":4593},[117],{"categories":4595},[117],{"categories":4597},[117],{"categories":4599},[188],{"categories":4601},[117],{"categories":4603},[117],{"categories":4605},[117],{"categories":4607},[3198],{"categories":4609},[4610],"Clinical AI",{"categories":4612},[276],{"categories":4614},[117],{"categories":4616},[117],{"categories":4618},[117],{"categories":4620},[117],{"categories":4622},[346],{"categories":4624},[2557],{"categories":4626},[117],{"categories":4628},[191],{"categories":4630},[276],{"categories":4632},[117],{"categories":4634},[188],{"categories":4636},[117],{"categories":4638},[117],{"categories":4640},[224],{"categories":4642},[117],{"categories":4644},[188],{"categories":4646},[202],{"categories":4648},[305],{"categories":4650},[117],{"categories":4652},[117],{"categories":4654},[183],{"categories":4656},[117],{"categories":4658},[117],{"categories":4660},[578],{"categories":4662},[117],{"categories":4664},[],{"categories":4666},[188],{"categories":4668},[117],{"categories":4670},[202],{"categories":4672},[178],{"categories":4674},[117],{"categories":4676},[],{"categories":4678},[],{"categories":4680},[117],{"categories":4682},[],{"categories":4684},[183],{"categories":4686},[117],{"categories":4688},[117],{"categories":4690},[188],{"categories":4692},[117],{"categories":4694},[224],{"categories":4696},[224],{"categories":4698},[224],{"categories":4700},[224],{"categories":4702},[],{"categories":4704},[178],{"categories":4706},[188],{"categories":4708},[224],{"categories":4710},[117],{"categories":4712},[649],{"categories":4714},[191],{"categories":4716},[188],{"categories":4718},[117],{"categories":4720},[178],{"categories":4722},[117],{"categories":4724},[188],{"categories":4726},[117],{"categories":4728},[117],{"categories":4730},[117],{"categories":4732},[117,188],{"categories":4734},[188],{"categories":4736},[346],{"categories":4738},[224],{"categories":4740},[188],{"categories":4742},[224],{"categories":4744},[188],{"categories":4746},[117],{"categories":4748},[],{"categories":4750},[224],{"categories":4752},[305],{"categories":4754},[178],{"categories":4756},[117],{"categories":4758},[117],{"categories":4760},[],{"categories":4762},[202],{"categories":4764},[],{"categories":4766},[178],{"categories":4768},[188],{"categories":4770},[224],{"categories":4772},[117],{"categories":4774},[224],{"categories":4776},[178],{"categories":4778},[224],{"categories":4780},[224],{"categories":4782},[],{"categories":4784},[183],{"categories":4786},[188],{"categories":4788},[224],{"categories":4790},[224],{"categories":4792},[224],{"categories":4794},[224],{"categories":4796},[224],{"categories":4798},[224],{"categories":4800},[224],{"categories":4802},[224],{"categories":4804},[224],{"categories":4806},[224],{"categories":4808},[227],{"categories":4810},[178],{"categories":4812},[117],{"categories":4814},[117],{"categories":4816},[188],{"categories":4818},[188],{"categories":4820},[],{"categories":4822},[117],{"categories":4824},[117,178],{"categories":4826},[],{"categories":4828},[188],{"categories":4830},[117],{"categories":4832},[224],{"categories":4834},[188],{"categories":4836},[974],{"categories":4838},[117],{"categories":4840},[117],{"categories":4842},[117],{"categories":4844},[117],{"categories":4846},[117],{"categories":4848},[440],{"categories":4850},[117],{"categories":4852},[117],{"categories":4854},[188],{"categories":4856},[117],{"categories":4858},[117],{"categories":4860},[183],{"categories":4862},[191],{"categories":4864},[188],{"categories":4866},[188],{"categories":4868},[],{"categories":4870},[188],{"categories":4872},[276],{"categories":4874},[224],{"categories":4876},[117],{"categories":4878},[],{"categories":4880},[191],{"categories":4882},[],{"categories":4884},[202],{"categories":4886},[117],{"categories":4888},[188],{"categories":4890},[276],{"categories":4892},[117],{"categories":4894},[],{"categories":4896},[117],{"categories":4898},[117],{"categories":4900},[],{"categories":4902},[305],{"categories":4904},[117],{"categories":4906},[188],{"categories":4908},[],{"categories":4910},[],{"categories":4912},[224],{"categories":4914},[178],{"categories":4916},[117],{"categories":4918},[117],{"categories":4920},[183],{"categories":4922},[117],{"categories":4924},[117],{"categories":4926},[188],{"categories":4928},[117],{"categories":4930},[183],{"categories":4932},[183],{"categories":4934},[276],{"categories":4936},[],{"categories":4938},[117],{"categories":4940},[224],{"categories":4942},[],{"categories":4944},[117],{"categories":4946},[117],{"categories":4948},[276],{"categories":4950},[117],{"categories":4952},[117],{"categories":4954},[305],{"categories":4956},[117],{"categories":4958},[346],{"categories":4960},[],{"categories":4962},[188],{"categories":4964},[117],{"categories":4966},[305],{"categories":4968},[202],{"categories":4970},[],{"categories":4972},[117],{"categories":4974},[],{"categories":4976},[188],{"categories":4978},[276],{"categories":4980},[202],{"categories":4982},[],{"categories":4984},[3198],{"categories":4986},[183],{"categories":4988},[178],{"categories":4990},[117],{"categories":4992},[227],{"categories":4994},[188],{"categories":4996},[276],{"categories":4998},[117],{"categories":5000},[202],{"categories":5002},[],{"categories":5004},[],{"categories":5006},[117],{"categories":5008},[178],{"categories":5010},[117],{"categories":5012},[305],{"categories":5014},[],{"categories":5016},[188],{"categories":5018},[188],{"categories":5020},[117],{"categories":5022},[188],{"categories":5024},[117],{"categories":5026},[224],{"categories":5028},[202],{"categories":5030},[117],{"categories":5032},[188],{"categories":5034},[191],{"categories":5036},[117],{"categories":5038},[117],{"categories":5040},[117],{"categories":5042},[188],{"categories":5044},[117],{"categories":5046},[191],{"categories":5048},[305],{"categories":5050},[224],{"categories":5052},[],{"categories":5054},[305],{"categories":5056},[117],{"categories":5058},[],{"categories":5060},[202],{"categories":5062},[188],{"categories":5064},[],{"categories":5066},[117],{"categories":5068},[117],{"categories":5070},[117],{"categories":5072},[117],{"categories":5074},[117],{"categories":5076},[188],{"categories":5078},[183],{"categories":5080},[178],{"categories":5082},[188],{"categories":5084},[117],{"categories":5086},[276],{"categories":5088},[202],{"categories":5090},[202],{"categories":5092},[117],{"categories":5094},[227],{"categories":5096},[188],{"categories":5098},[117],{"categories":5100},[117],{"categories":5102},[188],{"categories":5104},[117],{"categories":5106},[117],{"categories":5108},[188],{"categories":5110},[183],{"categories":5112},[117],{"categories":5114},[276],{"categories":5116},[202],{"categories":5118},[188],{"categories":5120},[117],{"categories":5122},[191],{"categories":5124},[117],{"categories":5126},[188],{"categories":5128},[117],{"categories":5130},[117],{"categories":5132},[224],{"categories":5134},[117],{"categories":5136},[],{"categories":5138},[178],{"categories":5140},[117],{"categories":5142},[117],{"categories":5144},[117],{"categories":5146},[202],{"categories":5148},[202],{"categories":5150},[117],{"categories":5152},[202],{"categories":5154},[117],{"categories":5156},[188],{"categories":5158},[117],{"categories":5160},[117],{"categories":5162},[117],{"categories":5164},[117],{"categories":5166},[117],{"categories":5168},[],{"categories":5170},[117],{"categories":5172},[276],{"categories":5174},[188],{"categories":5176},[183],{"categories":5178},[224],{"categories":5180},[117],{"categories":5182},[188],{"categories":5184},[117],{"categories":5186},[188],{"categories":5188},[117],{"categories":5190},[117],{"categories":5192},[276],{"categories":5194},[188],{"categories":5196},[117],{"categories":5198},[305],{"categories":5200},[117],{"categories":5202},[227],{"categories":5204},[117],{"categories":5206},[117],{"categories":5208},[224],{"categories":5210},[117],{"categories":5212},[117],{"categories":5214},[117],{"categories":5216},[117],{"categories":5218},[188],{"categories":5220},[346],{"categories":5222},[117],{"categories":5224},[202],{"categories":5226},[188],{"categories":5228},[227],{"categories":5230},[],{"categories":5232},[188],{"categories":5234},[202],{"categories":5236},[117],{"categories":5238},[117],{"categories":5240},[2395],{"categories":5242},[276],{"categories":5244},[375],{"categories":5246},[117],{"categories":5248},[117],{"categories":5250},[117],{"categories":5252},[117],{"categories":5254},[178],{"categories":5256},[117],{"categories":5258},[117],{"categories":5260},[202],{"categories":5262},[183],{"categories":5264},[117],{"categories":5266},[202],{"categories":5268},[117],{"categories":5270},[],{"categories":5272},[188],{"categories":5274},[188],{"categories":5276},[117],{"categories":5278},[117],{"categories":5280},[117],{"categories":5282},[227],{"categories":5284},[],{"categories":5286},[224],{"categories":5288},[],{"categories":5290},[224],{"categories":5292},[117],{"categories":5294},[117],{"categories":5296},[188],{"categories":5298},[117],{"categories":5300},[188],{"categories":5302},[188],{"categories":5304},[],{"categories":5306},[117],{"categories":5308},[224],{"categories":5310},[117],{"categories":5312},[],{"categories":5314},[117],{"categories":5316},[117],{"categories":5318},[],{"categories":5320},[117],{"categories":5322},[117],{"categories":5324},[276],{"categories":5326},[202],{"categories":5328},[188],{"categories":5330},[117],{"categories":5332},[117],{"categories":5334},[117],{"categories":5336},[117],{"categories":5338},[305],{"categories":5340},[117],{"categories":5342},[117],{"categories":5344},[117],{"categories":5346},[178],{"categories":5348},[117],{"categories":5350},[117],{"categories":5352},[],{"categories":5354},[117],{"categories":5356},[117],{"categories":5358},[117],{"categories":5360},[],{"categories":5362},[178],{"categories":5364},[117],{"categories":5366},[117],{"categories":5368},[224],{"categories":5370},[202],{"categories":5372},[191],{"categories":5374},[188],{"categories":5376},[519],{"categories":5378},[117],{"categories":5380},[117],{"categories":5382},[117],{"categories":5384},[202],{"categories":5386},[224],{"categories":5388},[276],{"categories":5390},[117],{"categories":5392},[117],{"categories":5394},[117],{"categories":5396},[117],{"categories":5398},[224],{"categories":5400},[117],{"categories":5402},[276],{"categories":5404},[117],{"categories":5406},[117],{"categories":5408},[224],{"categories":5410},[276],{"categories":5412},[117],{"categories":5414},[224],{"categories":5416},[117],{"categories":5418},[188],{"categories":5420},[188],{"categories":5422},[188],{"categories":5424},[202],{"categories":5426},[224],{"categories":5428},[188],{"categories":5430},[188],{"categories":5432},[117],{"categories":5434},[202],{"categories":5436},[276],{"categories":5438},[117],{"categories":5440},[117],{"categories":5442},[188],{"categories":5444},[117],{"categories":5446},[],{"categories":5448},[188],{"categories":5450},[],{"categories":5452},[117],{"categories":5454},[117],{"categories":5456},[],{"categories":5458},[],{"categories":5460},[188],{"categories":5462},[183],{"categories":5464},[188],{"categories":5466},[5467],"Liability & Ethics",{"categories":5469},[117],{"categories":5471},[117],{"categories":5473},[117],{"categories":5475},[188],{"categories":5477},[178],{"categories":5479},[188],{"categories":5481},[183],{"categories":5483},[305],{"categories":5485},[188],{"categories":5487},[117],{"categories":5489},[117],{"categories":5491},[],{"categories":5493},[624],{"categories":5495},[188],{"categories":5497},[],{"categories":5499},[117],{"categories":5501},[178],{"categories":5503},[188],{"categories":5505},[],{"categories":5507},[188],{"categories":5509},[117],{"categories":5511},[117],{"categories":5513},[202],{"categories":5515},[117],{"categories":5517},[224],{"categories":5519},[117],{"categories":5521},[117],{"categories":5523},[191],{"categories":5525},[188],{"categories":5527},[117],{"categories":5529},[117],{"categories":5531},[117],{"categories":5533},[224],{"categories":5535},[188],{"categories":5537},[202],{"categories":5539},[276],{"categories":5541},[178],{"categories":5543},[117],{"categories":5545},[117],{"categories":5547},[117],{"categories":5549},[],{"categories":5551},[188],{"categories":5553},[188],{"categories":5555},[188],{"categories":5557},[519],{"categories":5559},[276],{"categories":5561},[188],{"categories":5563},[346],{"categories":5565},[202],{"categories":5567},[224],{"categories":5569},[117],{"categories":5571},[276],{"categories":5573},[117],{"categories":5575},[178],{"categories":5577},[],{"categories":5579},[188],{"categories":5581},[117],{"categories":5583},[117],{"categories":5585},[117],{"categories":5587},[117],{"categories":5589},[188],{"categories":5591},[117],{"categories":5593},[117],{"categories":5595},[276],{"categories":5597},[],{"categories":5599},[188],{"categories":5601},[191],{"categories":5603},[224],{"categories":5605},[188],{"categories":5607},[183],{"categories":5609},[],{"categories":5611},[117],{"categories":5613},[117],{"categories":5615},[191],{"categories":5617},[117],{"categories":5619},[188],{"categories":5621},[224],{"categories":5623},[178],{"categories":5625},[346],{"categories":5627},[117],{"categories":5629},[117],{"categories":5631},[117],{"categories":5633},[224],{"categories":5635},[183],{"categories":5637},[117],{"categories":5639},[276],{"categories":5641},[224],{"categories":5643},[346],{"categories":5645},[117],{"categories":5647},[188],{"categories":5649},[],{"categories":5651},[578],{"categories":5653},[],{"categories":5655},[117],{"categories":5657},[346],{"categories":5659},[117],{"categories":5661},[227],{"categories":5663},[117],{"categories":5665},[188],{"categories":5667},[188],{"categories":5669},[5670],"Design News & Tools",{"categories":5672},[117],{"categories":5674},[117],{"categories":5676},[224],{"categories":5678},[117],{"categories":5680},[117],{"categories":5682},[178],{"categories":5684},[188],{"categories":5686},[117],{"categories":5688},[276],{"categories":5690},[188],{"categories":5692},[188],{"categories":5694},[276],{"categories":5696},[117],{"categories":5698},[117],{"categories":5700},[519],{"categories":5702},[188],{"categories":5704},[117],{"categories":5706},[117],{"categories":5708},[519],{"categories":5710},[117],{"categories":5712},[305],{"categories":5714},[117],{"categories":5716},[188],{"categories":5718},[],{"categories":5720},[117],{"categories":5722},[117],{"categories":5724},[117],{"categories":5726},[224],{"categories":5728},[117],{"categories":5730},[178],{"categories":5732},[],{"categories":5734},[117],{"categories":5736},[117],{"categories":5738},[117],{"categories":5740},[202],{"categories":5742},[649],{"categories":5744},[202],{"categories":5746},[276],{"categories":5748},[117],{"categories":5750},[117,188],{"categories":5752},[305,183],{"categories":5754},[202],{"categories":5756},[117],{"categories":5758},[117],{"categories":5760},[117],{"categories":5762},[117],{"categories":5764},[],{"categories":5766},[188],{"categories":5768},[117],{"categories":5770},[],{"categories":5772},[117],{"categories":5774},[202],{"categories":5776},[117],{"categories":5778},[202],{"categories":5780},[],{"categories":5782},[188],{"categories":5784},[117],{"categories":5786},[183],{"categories":5788},[117],{"categories":5790},[224],{"categories":5792},[117],{"categories":5794},[],{"categories":5796},[188],{"categories":5798},[117],{"categories":5800},[],{"categories":5802},[276],{"categories":5804},[117],{"categories":5806},[117],{"categories":5808},[188],{"categories":5810},[117],{"categories":5812},[117],{"categories":5814},[178],{"categories":5816},[188],{"categories":5818},[117],{"categories":5820},[],{"categories":5822},[117],{"categories":5824},[346],{"categories":5826},[305],{"categories":5828},[183],{"categories":5830},[183],{"categories":5832},[117],{"categories":5834},[178],{"categories":5836},[178],{"categories":5838},[117],{"categories":5840},[188],{"categories":5842},[117],{"categories":5844},[117],{"categories":5846},[117],{"categories":5848},[117],{"categories":5850},[202],{"categories":5852},[117],{"categories":5854},[178],{"categories":5856},[117],{"categories":5858},[117],{"categories":5860},[188],{"categories":5862},[117],{"categories":5864},[305],{"categories":5866},[117],{"categories":5868},[224],{"categories":5870},[117],{"categories":5872},[117],{"categories":5874},[188],{"categories":5876},[191],{"categories":5878},[117],{"categories":5880},[117],{"categories":5882},[188],{"categories":5884},[],{"categories":5886},[202],{"categories":5888},[],{"categories":5890},[202],{"categories":5892},[188],{"categories":5894},[178],{"categories":5896},[117],{"categories":5898},[],{"categories":5900},[227],{"categories":5902},[346],{"categories":5904},[117],{"categories":5906},[202],{"categories":5908},[117],{"categories":5910},[],{"categories":5912},[224],{"categories":5914},[188],{"categories":5916},[202],{"categories":5918},[276],{"categories":5920},[183],{"categories":5922},[117],{"categories":5924},[117],{"categories":5926},[188],{"categories":5928},[202],{"categories":5930},[188],{"categories":5932},[224],{"categories":5934},[117],{"categories":5936},[191],{"categories":5938},[178],{"categories":5940},[191],{"categories":5942},[224],{"categories":5944},[117],{"categories":5946},[202],{"categories":5948},[117],{"categories":5950},[276],{"categories":5952},[183],{"categories":5954},[117],{"categories":5956},[117],{"categories":5958},[117],{"categories":5960},[117],{"categories":5962},[117],{"categories":5964},[117],{"categories":5966},[188],{"categories":5968},[117],{"categories":5970},[188],{"categories":5972},[117],{"categories":5974},[117],{"categories":5976},[178],{"categories":5978},[117],{"categories":5980},[188],{"categories":5982},[188],{"categories":5984},[276],{"categories":5986},[188],{"categories":5988},[188],{"categories":5990},[117],{"categories":5992},[178],{"categories":5994},[188],{"categories":5996},[276],{"categories":5998},[],{"categories":6000},[117],{"categories":6002},[227],{"categories":6004},[519],{"categories":6006},[117],{"categories":6008},[188],{"categories":6010},[117],{"categories":6012},[117],{"categories":6014},[202],{"categories":6016},[117],{"categories":6018},[],{"categories":6020},[117],{"categories":6022},[188],{"categories":6024},[117],{"categories":6026},[305],{"categories":6028},[117],{"categories":6030},[202],{"categories":6032},[117],{"categories":6034},[224],{"categories":6036},[188],{"categories":6038},[117],{"categories":6040},[305],{"categories":6042},[188],{"categories":6044},[183],{"categories":6046},[183],{"categories":6048},[117],{"categories":6050},[117],{"categories":6052},[117],{"categories":6054},[117],{"categories":6056},[117],{"categories":6058},[117],{"categories":6060},[178],{"categories":6062},[],{"categories":6064},[117],{"categories":6066},[117],{"categories":6068},[188],{"categories":6070},[188],{"categories":6072},[117],{"categories":6074},[117],{"categories":6076},[117],{"categories":6078},[117],{"categories":6080},[117],{"categories":6082},[202],{"categories":6084},[],{"categories":6086},[178],{"categories":6088},[117],{"categories":6090},[117],{"categories":6092},[188],{"categories":6094},[188],{"categories":6096},[],{"categories":6098},[202],{"categories":6100},[202],{"categories":6102},[117],{"categories":6104},[305],{"categories":6106},[183],{"categories":6108},[276],{"categories":6110},[],{"categories":6112},[117],{"categories":6114},[188],{"categories":6116},[178],{"categories":6118},[117],{"categories":6120},[117],{"categories":6122},[202],{"categories":6124},[178],{"categories":6126},[117],{"categories":6128},[117],{"categories":6130},[224],{"categories":6132},[227],{"categories":6134},[117],{"categories":6136},[224],{"categories":6138},[188],{"categories":6140},[117],{"categories":6142},[],{"categories":6144},[224],{"categories":6146},[188],{"categories":6148},[276],{"categories":6150},[227],{"categories":6152},[117],{"categories":6154},[117],{"categories":6156},[],{"categories":6158},[188],{"categories":6160},[188],{"categories":6162},[188],{"categories":6164},[3198],{"categories":6166},[224],{"categories":6168},[117],{"categories":6170},[202],{"categories":6172},[117],{"categories":6174},[117],{"categories":6176},[117],{"categories":6178},[117],{"categories":6180},[117],{"categories":6182},[183],{"categories":6184},[117],{"categories":6186},[178],{"categories":6188},[1861],{"categories":6190},[346],{"categories":6192},[178],{"categories":6194},[],{"categories":6196},[117],{"categories":6198},[],{"categories":6200},[224],{"categories":6202},[188],{"categories":6204},[276],{"categories":6206},[117],{"categories":6208},[117],{"categories":6210},[117],{"categories":6212},[224],{"categories":6214},[],{"categories":6216},[188],{"categories":6218},[117],{"categories":6220},[188],{"categories":6222},[188],{"categories":6224},[],{"categories":6226},[117],{"categories":6228},[],{"categories":6230},[224],{"categories":6232},[178],{"categories":6234},[276],{"categories":6236},[117],{"categories":6238},[188],{"categories":6240},[224],{"categories":6242},[117],{"categories":6244},[224],{"categories":6246},[],{"categories":6248},[224],{"categories":6250},[117],{"categories":6252},[178],{"categories":6254},[519],{"categories":6256},[188],{"categories":6258},[117],{"categories":6260},[],{"categories":6262},[202],{"categories":6264},[188],{"categories":6266},[191],{"categories":6268},[188],{"categories":6270},[178],{"categories":6272},[117],{"categories":6274},[117],{"categories":6276},[],{"categories":6278},[],{"categories":6280},[],{"categories":6282},[276],{"categories":6284},[117],{"categories":6286},[188],{"categories":6288},[117],{"categories":6290},[117],{"categories":6292},[],{"categories":6294},[],{"categories":6296},[],{"categories":6298},[117],{"categories":6300},[188],{"categories":6302},[276],{"categories":6304},[117],{"categories":6306},[],{"categories":6308},[188],{"categories":6310},[117],{"categories":6312},[117],{"categories":6314},[178],{"categories":6316},[],{"categories":6318},[],{"categories":6320},[117],{"categories":6322},[117],{"categories":6324},[188],{"categories":6326},[276],{"categories":6328},[117],{"categories":6330},[224],{"categories":6332},[],{"categories":6334},[117],{"categories":6336},[117],{"categories":6338},[305],{"categories":6340},[224],{"categories":6342},[305],{"categories":6344},[227],{"categories":6346},[117],{"categories":6348},[117],{"categories":6350},[],{"categories":6352},[],{"categories":6354},[188],{"categories":6356},[],{"categories":6358},[117],{"categories":6360},[519],{"categories":6362},[117],{"categories":6364},[117],{"categories":6366},[117],{"categories":6368},[117],{"categories":6370},[],{"categories":6372},[188],{"categories":6374},[117],{"categories":6376},[117],{"categories":6378},[],{"categories":6380},[188],{"categories":6382},[117],{"categories":6384},[224],{"categories":6386},[117],{"categories":6388},[305],{"categories":6390},[183],{"categories":6392},[191],{"categories":6394},[117],{"categories":6396},[117],{"categories":6398},[188],{"categories":6400},[227],{"categories":6402},[188],{"categories":6404},[188],{"categories":6406},[],{"categories":6408},[117],{"categories":6410},[188],{"categories":6412},[],{"categories":6414},[117],{"categories":6416},[],{"categories":6418},[224],{"categories":6420},[183],{"categories":6422},[],{"categories":6424},[117],{"categories":6426},[117],{"categories":6428},[117],{"categories":6430},[],{"categories":6432},[188],{"categories":6434},[276],{"categories":6436},[178],{"categories":6438},[117],{"categories":6440},[],{"categories":6442},[183],{"categories":6444},[305],{"categories":6446},[117],{"categories":6448},[202],{"categories":6450},[178],{"categories":6452},[227],{"categories":6454},[183],{"categories":6456},[202],{"categories":6458},[188],{"categories":6460},[202],{"categories":6462},[],{"categories":6464},[117],{"categories":6466},[191],{"categories":6468},[117],{"categories":6470},[],{"categories":6472},[188],{"categories":6474},[178],{"categories":6476},[276],{"categories":6478},[117],{"categories":6480},[178],{"categories":6482},[188],{"categories":6484},[346],{"categories":6486},[117],{"categories":6488},[117],{"categories":6490},[117],{"categories":6492},[117],{"categories":6494},[117],{"categories":6496},[178],{"categories":6498},[117],{"categories":6500},[202],{"categories":6502},[227],{"categories":6504},[188],{"categories":6506},[],{"categories":6508},[117],{"categories":6510},[117],{"categories":6512},[117],{"categories":6514},[202],{"categories":6516},[188],{"categories":6518},[224],{"categories":6520},[202],{"categories":6522},[117],{"categories":6524},[191],{"categories":6526},[],{"categories":6528},[276],{"categories":6530},[202],{"categories":6532},[224],{"categories":6534},[117],{"categories":6536},[178],{"categories":6538},[188],{"categories":6540},[117],{"categories":6542},[117],{"categories":6544},[188],{"categories":6546},[191],{"categories":6548},[117],{"categories":6550},[188],{"categories":6552},[117],{"categories":6554},[183],{"categories":6556},[188],{"categories":6558},[188,346],{"categories":6560},[117],{"categories":6562},[117],{"categories":6564},[188],{"categories":6566},[202],{"categories":6568},[117],{"categories":6570},[117],{"categories":6572},[227],{"categories":6574},[188],{"categories":6576},[305],{"categories":6578},[188],{"categories":6580},[183],{"categories":6582},[],{"categories":6584},[188],{"categories":6586},[117],{"categories":6588},[183],{"categories":6590},[],{"categories":6592},[],{"categories":6594},[202],{"categories":6596},[117],{"categories":6598},[117],{"categories":6600},[188],{"categories":6602},[227],{"categories":6604},[305],{"categories":6606},[117],{"categories":6608},[117],{"categories":6610},[117],{"categories":6612},[188],{"categories":6614},[],{"categories":6616},[188],{"categories":6618},[224],{"categories":6620},[117],{"categories":6622},[188],{"categories":6624},[188],{"categories":6626},[117],{"categories":6628},[],{"categories":6630},[224],{"categories":6632},[202],{"categories":6634},[3198],{"categories":6636},[178],{"categories":6638},[202],{"categories":6640},[117],{"categories":6642},[188],{"categories":6644},[117],{"categories":6646},[117],{"categories":6648},[305],{"categories":6650},[202],{"categories":6652},[227],{"categories":6654},[],{"categories":6656},[224],{"categories":6658},[117],{"categories":6660},[117],{"categories":6662},[],{"categories":6664},[188],{"categories":6666},[117],{"categories":6668},[117],{"categories":6670},[117],{"categories":6672},[117],{"categories":6674},[188],{"categories":6676},[117],{"categories":6678},[117],{"categories":6680},[117],{"categories":6682},[191],{"categories":6684},[117],{"categories":6686},[188],{"categories":6688},[117],{"categories":6690},[117],{"categories":6692},[117],{"categories":6694},[117],{"categories":6696},[117],{"categories":6698},[117],{"categories":6700},[117],{"categories":6702},[183],{"categories":6704},[],{"categories":6706},[191],{"categories":6708},[224],{"categories":6710},[188],{"categories":6712},[117],{"categories":6714},[202],{"categories":6716},[],{"categories":6718},[202],{"categories":6720},[202],{"categories":6722},[188],{"categories":6724},[202],{"categories":6726},[117],{"categories":6728},[117],{"categories":6730},[117],{"categories":6732},[188],{"categories":6734},[202],{"categories":6736},[117],{"categories":6738},[117],{"categories":6740},[117],{"categories":6742},[188],{"categories":6744},[224],{"categories":6746},[117],{"categories":6748},[117],{"categories":6750},[117],{"categories":6752},[183],{"categories":6754},[117],{"categories":6756},[188],{"categories":6758},[276],{"categories":6760},[],{"categories":6762},[117],{"categories":6764},[227],{"categories":6766},[188],{"categories":6768},[117],{"categories":6770},[117],{"categories":6772},[],{"categories":6774},[117],{"categories":6776},[117],{"categories":6778},[224],{"categories":6780},[117],{"categories":6782},[117],{"categories":6784},[188],{"categories":6786},[305],{"categories":6788},[],{"categories":6790},[],{"categories":6792},[202],{"categories":6794},[117],{"categories":6796},[117],{"categories":6798},[224],{"categories":6800},[117],{"categories":6802},[202],{"categories":6804},[224],{"categories":6806},[117],{"categories":6808},[117],{"categories":6810},[305],{"categories":6812},[227],{"categories":6814},[117],{"categories":6816},[117],{"categories":6818},[178],{"categories":6820},[188],{"categories":6822},[117],{"categories":6824},[117],{"categories":6826},[188],{"categories":6828},[183],{"categories":6830},[188],{"categories":6832},[202],{"categories":6834},[117],{"categories":6836},[183],{"categories":6838},[],{"categories":6840},[117],{"categories":6842},[227],{"categories":6844},[117],{"categories":6846},[117],{"categories":6848},[],{"categories":6850},[224],{"categories":6852},[117],{"categories":6854},[188],{"categories":6856},[227],{"categories":6858},[117],{"categories":6860},[202],{"categories":6862},[202],{"categories":6864},[202],{"categories":6866},[117],{"categories":6868},[188],{"categories":6870},[188],{"categories":6872},[117],{"categories":6874},[188],{"categories":6876},[117],{"categories":6878},[117],{"categories":6880},[276],{"categories":6882},[227],{"categories":6884},[227],{"categories":6886},[],{"categories":6888},[224],{"categories":6890},[117],{"categories":6892},[117],{"categories":6894},[202],{"categories":6896},[],{"categories":6898},[224],{"categories":6900},[224],{"categories":6902},[224],{"categories":6904},[],{"categories":6906},[188],{"categories":6908},[117],{"categories":6910},[],{"categories":6912},[178],{"categories":6914},[183],{"categories":6916},[],{"categories":6918},[117],{"categories":6920},[117],{"categories":6922},[],{"categories":6924},[202],{"categories":6926},[],{"categories":6928},[],{"categories":6930},[],{"categories":6932},[],{"categories":6934},[117],{"categories":6936},[224],{"categories":6938},[],{"categories":6940},[],{"categories":6942},[117],{"categories":6944},[117],{"categories":6946},[117],{"categories":6948},[227],{"categories":6950},[117],{"categories":6952},[227],{"categories":6954},[],{"categories":6956},[227],{"categories":6958},[227],{"categories":6960},[346],{"categories":6962},[188],{"categories":6964},[202],{"categories":6966},[],{"categories":6968},[],{"categories":6970},[227],{"categories":6972},[202],{"categories":6974},[202],{"categories":6976},[202],{"categories":6978},[],{"categories":6980},[178],{"categories":6982},[202],{"categories":6984},[202],{"categories":6986},[178],{"categories":6988},[202],{"categories":6990},[183],{"categories":6992},[202],{"categories":6994},[202],{"categories":6996},[202],{"categories":6998},[227],{"categories":7000},[224],{"categories":7002},[224],{"categories":7004},[117],{"categories":7006},[202],{"categories":7008},[227],{"categories":7010},[346],{"categories":7012},[227],{"categories":7014},[227],{"categories":7016},[227],{"categories":7018},[],{"categories":7020},[183],{"categories":7022},[],{"categories":7024},[346],{"categories":7026},[202],{"categories":7028},[202],{"categories":7030},[202],{"categories":7032},[188],{"categories":7034},[224,183],{"categories":7036},[227],{"categories":7038},[],{"categories":7040},[],{"categories":7042},[227],{"categories":7044},[],{"categories":7046},[227],{"categories":7048},[224],{"categories":7050},[188],{"categories":7052},[],{"categories":7054},[202],{"categories":7056},[117],{"categories":7058},[276],{"categories":7060},[],{"categories":7062},[117],{"categories":7064},[],{"categories":7066},[224],{"categories":7068},[178],{"categories":7070},[227],{"categories":7072},[],{"categories":7074},[202],{"categories":7076},[224],[7078,7170,7246,7303],{"id":7079,"title":7080,"ai":7081,"body":7086,"categories":7140,"created_at":118,"date_modified":118,"description":110,"extension":119,"faq":118,"featured":120,"kicker_label":118,"meta":7141,"navigation":152,"path":7157,"published_at":7158,"question":118,"scraped_at":7158,"seo":7159,"sitemap":7160,"source_id":7161,"source_name":7162,"source_type":7163,"source_url":7164,"stem":7165,"tags":7166,"thumbnail_url":118,"tldr":7167,"tweet":118,"unknown_tags":7168,"__hash__":7169},"summaries\u002Fsummaries\u002F41e798aa1961a662-beyond-accuracy-evaluating-ai-agents-after-benchma-summary.md","Beyond Accuracy: Evaluating AI Agents After Benchmark Saturation",{"provider":7,"model":8,"input_tokens":7082,"output_tokens":7083,"processing_time_ms":7084,"cost_usd":7085},6281,620,3629,0.00250025,{"type":14,"value":7087,"toc":7135},[7088,7092,7095,7099,7102,7128,7132],[17,7089,7091],{"id":7090},"the-problem-with-accuracy-centric-evaluation","The Problem with Accuracy-Centric Evaluation",[22,7093,7094],{},"Traditional AI evaluation often treats benchmarks as binary: once a model reaches high accuracy, the benchmark is considered 'solved' and retired. This approach is fundamentally flawed because it ignores the nuances of how agents actually perform in real-world environments. Relying solely on accuracy masks critical failures in reasoning, efficiency, and robustness that only become apparent when we look beyond the final score.",[17,7096,7098],{"id":7097},"a-multi-dimensional-evaluation-framework","A Multi-Dimensional Evaluation Framework",[22,7100,7101],{},"Using CORE-Bench Hard as a case study, the authors propose shifting focus toward six key dimensions of agent performance that remain relevant even after accuracy saturates:",[26,7103,7104,7110,7116,7122],{},[29,7105,7106,7109],{},[32,7107,7108],{},"Construct Validity:"," Identifying 'shortcuts' where agents pass tests without true understanding. The authors introduced CORE-Bench v1.1 and an out-of-distribution (OOD) suite to better stress-test agent capabilities.",[29,7111,7112,7115],{},[32,7113,7114],{},"Efficiency & Reliability:"," Measuring the cost, time, and consistency of agent outputs rather than just the correctness of the final result.",[29,7117,7118,7121],{},[32,7119,7120],{},"Model vs. Scaffold Performance:"," Disentangling the intelligence of the underlying LLM from the effectiveness of the 'scaffold' (the code\u002Fsystem wrapping the model).",[29,7123,7124,7127],{},[32,7125,7126],{},"Human-Agent Collaboration:"," Assessing the 'uplift' provided by AI in real-world workflows. In a randomized experiment on computational reproducibility, human-agent teams achieved a 2x speedup compared to humans alone, though the study noted that many human-only attempts hit time limits, suggesting the true performance gap may be even wider.",[17,7129,7131],{"id":7130},"moving-toward-rigorous-benchmarking","Moving Toward Rigorous Benchmarking",[22,7133,7134],{},"The authors argue that saturation is not an end-point but an opportunity to deepen evaluation. By moving away from simple accuracy metrics, developers can build more reliable systems that are actually useful in production. The findings demonstrate that even when agents appear to 'solve' a task, they often lack the reliability and efficiency required for professional scientific or engineering work. Future benchmarking should prioritize these operational metrics to ensure AI tools are ready for real-world deployment.",{"title":110,"searchDepth":111,"depth":111,"links":7136},[7137,7138,7139],{"id":7090,"depth":111,"text":7091},{"id":7097,"depth":111,"text":7098},{"id":7130,"depth":111,"text":7131},[117],{"content_references":7142,"triage":7153},[7143,7146,7149,7151],{"type":7144,"title":7145,"context":134},"other","CORE-Bench",{"type":7144,"title":7147,"context":7148},"CORE-Bench Hard","reviewed",{"type":7144,"title":7150,"context":127},"CORE-Bench v1.1",{"type":7144,"title":7152,"context":127},"CORE-Bench OOD",{"relevance":7154,"novelty":148,"quality":148,"actionability":149,"composite":7155,"reasoning":7156},5,4.15,"Category: AI & LLMs. The article provides a comprehensive framework for evaluating AI agents beyond traditional accuracy metrics, addressing a critical pain point for developers who need to ensure their AI systems are reliable and efficient in production. It introduces six dimensions of evaluation that can be directly applied to improve AI product development.","\u002Fsummaries\u002F41e798aa1961a662-beyond-accuracy-evaluating-ai-agents-after-benchma-summary","2026-06-26 12:58:18",{"title":7080,"description":110},{"loc":7157},"41e798aa1961a662","arXiv cs.AI","article","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.26158","summaries\u002F41e798aa1961a662-beyond-accuracy-evaluating-ai-agents-after-benchma-summary",[164,165,166,167],"When AI benchmarks saturate, accuracy becomes a poor metric. Researchers should instead evaluate agents across six dimensions: construct validity, generalizability, efficiency, reliability, model\u002Fscaffold performance, and human-agent collaboration.",[166,167],"1Ugd05Q41bU5Qhe6sZOAaduce3Pr8FzheR7WbGXcis8",{"id":7171,"title":7172,"ai":7173,"body":7178,"categories":7226,"created_at":118,"date_modified":118,"description":110,"extension":119,"faq":118,"featured":120,"kicker_label":118,"meta":7227,"navigation":152,"path":7236,"published_at":7237,"question":118,"scraped_at":7237,"seo":7238,"sitemap":7239,"source_id":7240,"source_name":7162,"source_type":7163,"source_url":7232,"stem":7241,"tags":7242,"thumbnail_url":118,"tldr":7243,"tweet":118,"unknown_tags":7244,"__hash__":7245},"summaries\u002Fsummaries\u002F4584088cdc666f51-litreview-arena-benchmarking-ai-agents-for-literat-summary.md","LitReview Arena: Benchmarking AI Agents for Literature Synthesis",{"provider":7,"model":8,"input_tokens":7174,"output_tokens":7175,"processing_time_ms":7176,"cost_usd":7177},4032,502,3339,0.001761,{"type":14,"value":7179,"toc":7221},[7180,7184,7187,7191,7194,7214,7218],[17,7181,7183],{"id":7182},"the-challenge-of-evaluating-ai-driven-literature-reviews","The Challenge of Evaluating AI-Driven Literature Reviews",[22,7185,7186],{},"Evaluating AI agents tasked with academic literature reviews is notoriously difficult due to the subjective nature of synthesis, the risk of hallucinations, and the need for high-fidelity citation accuracy. Traditional static benchmarks often fail to capture the nuance required for scholarly work, where an agent must not only retrieve relevant papers but also synthesize conflicting findings and maintain strict adherence to source material.",[17,7188,7190],{"id":7189},"the-litreview-arena-framework","The LitReview Arena Framework",[22,7192,7193],{},"LitReview Arena addresses these gaps by implementing a battle-style, peer-review-inspired evaluation platform. By pitting different AI agents against each other in head-to-head comparisons, the framework allows for a dynamic assessment of performance. Human experts or high-performing models act as judges to score outputs based on specific criteria:",[26,7195,7196,7202,7208],{},[29,7197,7198,7201],{},[32,7199,7200],{},"Synthesis Quality:"," How well the agent integrates multiple sources rather than just summarizing them in isolation.",[29,7203,7204,7207],{},[32,7205,7206],{},"Citation Integrity:"," The ability to map claims directly to the correct source without hallucinating references.",[29,7209,7210,7213],{},[32,7211,7212],{},"Breadth and Depth:"," The agent's ability to identify seminal works while filtering out noise in a specific research domain.",[17,7215,7217],{"id":7216},"shifting-from-static-to-comparative-evaluation","Shifting from Static to Comparative Evaluation",[22,7219,7220],{},"By moving away from static datasets, LitReview Arena creates a competitive environment that forces agents to improve on real-world research tasks. The platform emphasizes that effective literature review agents must demonstrate 'scholarly reasoning'—the ability to identify gaps in existing literature and frame new research questions based on the synthesis of prior work. This approach provides a more robust signal for developers looking to build agents that can reliably assist in the scientific discovery process.",{"title":110,"searchDepth":111,"depth":111,"links":7222},[7223,7224,7225],{"id":7182,"depth":111,"text":7183},{"id":7189,"depth":111,"text":7190},{"id":7216,"depth":111,"text":7217},[117],{"content_references":7228,"triage":7233},[7229],{"type":7230,"title":7231,"url":7232,"context":7148},"paper","LitReview Arena: Evaluating Literature Review Agents with Battle-Style Peer Review Platform","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.21374",{"relevance":149,"novelty":148,"quality":148,"actionability":111,"composite":7234,"reasoning":7235},3.25,"Category: AI & LLMs. The article discusses a new framework for evaluating AI agents in the context of literature reviews, which is relevant to AI engineering and the development of AI-powered products. While it presents novel insights into benchmarking AI agents, it lacks specific actionable steps for developers looking to implement these findings in their own projects.","\u002Fsummaries\u002F4584088cdc666f51-litreview-arena-benchmarking-ai-agents-for-literat-summary","2026-08-26 03:10:17",{"title":7172,"description":110},{"loc":7236},"4584088cdc666f51","summaries\u002F4584088cdc666f51-litreview-arena-benchmarking-ai-agents-for-literat-summary",[164,165,166],"LitReview Arena introduces a battle-style evaluation platform to measure the accuracy, synthesis capabilities, and citation integrity of AI agents performing academic literature reviews.",[166],"Z9LUPyTjZmCJMYgMrDpV-sHHuRnbV8tAqcmVpiR8l-8",{"id":7247,"title":7248,"ai":7249,"body":7254,"categories":7285,"created_at":118,"date_modified":118,"description":110,"extension":119,"faq":118,"featured":120,"kicker_label":118,"meta":7286,"navigation":152,"path":7293,"published_at":7294,"question":118,"scraped_at":7294,"seo":7295,"sitemap":7296,"source_id":7297,"source_name":7162,"source_type":7163,"source_url":7290,"stem":7298,"tags":7299,"thumbnail_url":118,"tldr":7300,"tweet":118,"unknown_tags":7301,"__hash__":7302},"summaries\u002Fsummaries\u002F970bf1af87f06c39-consilience-improving-multi-agent-reasoning-via-ca-summary.md","Consilience: Improving Multi-Agent Reasoning via Calibration",{"provider":7,"model":8,"input_tokens":7250,"output_tokens":7251,"processing_time_ms":7252,"cost_usd":7253},4065,490,3072,0.00175125,{"type":14,"value":7255,"toc":7280},[7256,7260,7263,7267,7270,7273,7277],[17,7257,7259],{"id":7258},"addressing-the-hidden-profile-problem-in-multi-agent-systems","Addressing the Hidden-Profile Problem in Multi-Agent Systems",[22,7261,7262],{},"Multi-agent systems often struggle with \"hidden-profile\" scenarios, where individual agents possess incomplete, private information that must be synthesized to reach a correct global conclusion. Standard communication protocols often lead to confirmation bias or the premature convergence on incorrect solutions because agents fail to effectively share or weigh their unique, non-overlapping data. The Consilience framework addresses this by implementing a communication control mechanism that treats agent interactions as a statistical calibration problem.",[17,7264,7266],{"id":7265},"conformal-calibration-for-communication-control","Conformal Calibration for Communication Control",[22,7268,7269],{},"At the heart of Consilience is the use of conformal calibration to regulate what and when agents communicate. Instead of allowing agents to broadcast all information—which can lead to noise and cognitive overload—or restricting communication arbitrarily, the framework uses a calibration layer to assess the reliability and necessity of information exchange.",[22,7271,7272],{},"By applying conformal prediction techniques, the system ensures that the information shared by an agent is statistically likely to be relevant to the collective goal. This process effectively filters out redundant or misleading signals that typically derail multi-agent reasoning. The framework forces agents to maintain a degree of uncertainty until sufficient evidence is aggregated, preventing the group from locking into a consensus based on incomplete or biased initial observations.",[17,7274,7276],{"id":7275},"impact-on-reasoning-and-convergence","Impact on Reasoning and Convergence",[22,7278,7279],{},"The paper demonstrates that by controlling the flow of information through this calibrated approach, agents achieve higher accuracy in complex decision-making tasks. The framework provides a formal mechanism to balance the trade-off between communication overhead and reasoning depth. By ensuring that agents only communicate when the potential for information gain exceeds a calibrated threshold, the system avoids the common pitfalls of groupthink and information cascade, leading to more robust, data-driven outcomes in distributed reasoning environments.",{"title":110,"searchDepth":111,"depth":111,"links":7281},[7282,7283,7284],{"id":7258,"depth":111,"text":7259},{"id":7265,"depth":111,"text":7266},{"id":7275,"depth":111,"text":7276},[117],{"content_references":7287,"triage":7291},[7288],{"type":7230,"title":7289,"url":7290,"context":7148},"Consilience: Conformally Calibrated Communication Control for Hidden-Profile Multi-Agent Reasoning","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.20564",{"relevance":149,"novelty":148,"quality":148,"actionability":111,"composite":7234,"reasoning":7292},"Category: AI & LLMs. The article discusses a framework for improving multi-agent reasoning, which is relevant to AI engineering and addresses a specific challenge in AI systems. However, while it presents novel insights into calibration techniques, it lacks practical steps that the audience can directly implement.","\u002Fsummaries\u002F970bf1af87f06c39-consilience-improving-multi-agent-reasoning-via-ca-summary","2026-08-25 03:09:44",{"title":7248,"description":110},{"loc":7293},"970bf1af87f06c39","summaries\u002F970bf1af87f06c39-consilience-improving-multi-agent-reasoning-via-ca-summary",[164,165,166],"Consilience introduces a framework for multi-agent systems to solve hidden-profile problems by using conformal calibration to control communication and reduce information bias.",[166],"loxcPU73qYT6wW4Dl934r4MsY7zyf--lmajprzfnbOQ",{"id":7304,"title":7305,"ai":7306,"body":7311,"categories":7379,"created_at":118,"date_modified":118,"description":110,"extension":119,"faq":118,"featured":120,"kicker_label":118,"meta":7380,"navigation":152,"path":7388,"published_at":7389,"question":118,"scraped_at":7389,"seo":7390,"sitemap":7391,"source_id":7392,"source_name":7162,"source_type":7163,"source_url":7385,"stem":7393,"tags":7394,"thumbnail_url":118,"tldr":7395,"tweet":118,"unknown_tags":7396,"__hash__":7397},"summaries\u002Fsummaries\u002F9f38e290485b966c-the-limitations-of-agent-memory-in-tracking-evolvi-summary.md","The Limitations of Agent Memory in Tracking Evolving States",{"provider":7,"model":8,"input_tokens":7307,"output_tokens":7308,"processing_time_ms":7309,"cost_usd":7310},3991,575,3486,0.00186025,{"type":14,"value":7312,"toc":7374},[7313,7317,7320,7324,7327,7347,7351,7354],[17,7314,7316],{"id":7315},"the-state-tracking-problem-in-agentic-systems","The State Tracking Problem in Agentic Systems",[22,7318,7319],{},"Most contemporary AI agent memory architectures rely on retrieval-augmented generation (RAG) or vector-based semantic search to provide context. While these methods excel at retrieving static knowledge, they frequently fail when tasked with tracking 'evolving state'—the dynamic variables of an environment that change over time. The core issue is that semantic similarity, the primary mechanism for memory retrieval, is often decoupled from temporal relevance. An agent may retrieve a highly relevant document about a system's configuration that is factually outdated, leading to hallucinations or incorrect decision-making based on stale information.",[17,7321,7323],{"id":7322},"why-static-retrieval-fails","Why Static Retrieval Fails",[22,7325,7326],{},"Existing memory systems often treat state as a collection of independent documents rather than a continuous, mutable timeline. This leads to three primary failure modes:",[89,7328,7329,7335,7341],{},[29,7330,7331,7334],{},[32,7332,7333],{},"Temporal Aliasing:"," The system retrieves multiple conflicting versions of the same state parameter, with no clear mechanism to prioritize the most recent update.",[29,7336,7337,7340],{},[32,7338,7339],{},"State Fragmentation:"," Because state information is often stored across disparate chunks, the agent fails to reconstruct a coherent, global view of the current environment.",[29,7342,7343,7346],{},[32,7344,7345],{},"Update Latency:"," The overhead of re-indexing or updating vector databases often prevents real-time state tracking, forcing agents to operate on 'laggy' memory.",[17,7348,7350],{"id":7349},"moving-toward-dynamic-state-management","Moving Toward Dynamic State Management",[22,7352,7353],{},"To overcome these limitations, the research suggests that agent architectures must move beyond simple semantic retrieval. Effective state tracking requires:",[26,7355,7356,7362,7368],{},[29,7357,7358,7361],{},[32,7359,7360],{},"Explicit State Buffers:"," Maintaining a dedicated, structured memory buffer for mutable variables that is updated via transactional operations rather than semantic indexing.",[29,7363,7364,7367],{},[32,7365,7366],{},"Temporal Metadata:"," Attaching strict timestamps and versioning to all memory entries to allow for chronological filtering during the retrieval process.",[29,7369,7370,7373],{},[32,7371,7372],{},"State Reconciliation:"," Implementing a reasoning layer that explicitly checks for contradictions between retrieved 'historical' data and the 'current' state, forcing the agent to resolve conflicts before executing actions.",{"title":110,"searchDepth":111,"depth":111,"links":7375},[7376,7377,7378],{"id":7315,"depth":111,"text":7316},{"id":7322,"depth":111,"text":7323},{"id":7349,"depth":111,"text":7350},[117],{"content_references":7381,"triage":7386},[7382],{"type":7230,"title":7383,"author":7384,"url":7385,"context":7148},"Can Agent Memory Systems Track Evolving State?","Unknown","https:\u002F\u002Farxiv.org\u002Fabs\u002F2608.19652",{"relevance":148,"novelty":148,"quality":148,"actionability":149,"composite":150,"reasoning":7387},"Category: AI & LLMs. The article addresses the limitations of current agent memory systems, which is a core topic for AI product builders. It provides insights into specific failure modes and suggests improvements, making it relevant and actionable, though it lacks detailed frameworks for implementation.","\u002Fsummaries\u002F9f38e290485b966c-the-limitations-of-agent-memory-in-tracking-evolvi-summary","2026-08-22 03:10:21",{"title":7305,"description":110},{"loc":7388},"9f38e290485b966c","summaries\u002F9f38e290485b966c-the-limitations-of-agent-memory-in-tracking-evolvi-summary",[164,165,166],"Current agent memory systems struggle to maintain accurate, up-to-date state representations as environments change, often relying on static snapshots that fail to reflect temporal evolution.",[166],"jfIPg9ajg6x9xAlQ-j1oRij8k7b_-6oqkVu6JNHHUnE"]