According to Apple's release notes for the iOS 7 beta, the absence of the Voice Memos app is a known issue indicating that the app will return in future betas.
Voice Memos Known Issues ● The VoiceMemos app is not available in this seed.
Underscoring this, a leaked image from a WWDC developer session has surfaced online that shows an iOS 7 Springboard with the Voice Memos app present.